The Future of ECM

TSG is hosting a breakfast meeting with Doculabs on October 15^th in Rosemont, IL on the Future of Enterprise Content Management (ECM).  If you are interested in attending, please send us a note at inquiry@tsgrp.com.   This post will summarize the 20 minute TED format talk I am giving to kick-off the meeting.

What about the past of ECM?

Before going into the Future of ECM, it is important to understand the history of ECM.  Early image systems first emerged in the late 1980s with the advent of both the Workstation/PC and the network.  The original dominant vendor was FileNet with a focus on image processing.  For those that remember early FileNet or the other vendors, the original solution typically included:

• Specialized Viewing and Image Scanning Hardware – Typically this was UNIX workstations or PC’s with specialized boards.
• Large Monitors – Many of the solutions I was involved with focused on specialized dual-page monitors, requiring a very deep desk.
• Specialized Server Hardware – Initially FileNet provided proprietary servers (glorified Macs).  These were quickly replaced with UNIX/DEC servers and eventually displaced by Windows or Linux.
• Specialized Storage – See our post on a recent FileNet migration but, for a while the laser disc was the optimal cost-effective storage device. Typically most image systems involved a jukebox for storing and accessing the different platters.
• Database and File Pointers – At the core of any of the image solutions was a database (FileNet picked Oracle) with pointers to a file system (or the optical jukebox).
• ECM Software – Typically written above the database to offer API services with interfaces to the integrated solution.

By the end of the 1990’s, the stack of our ECM solution had evolved to include:

• Viewing and Scanning – by 1993 powerful PC’s and processors as well as standards to image scanning disrupted specialized components.
• Monitors – By the mid-90’s, large monitors were disrupted by normal monitors and later by LCD’s.
• Optical Drives – These have been long replaced with spinning disk magnetic and more recently memory drives. The storage architecture has been replaced with commercial Storage Array Networks (SAN).

While updated, the ECM Software (FileNet/Documentum) as well as database components evolved with technology innovations but were not disrupted with better and cheaper alternatives.

Technology Disruption in the 2000s

Technology Disruptions occurs when a new technology comes around that offers new better/quicker/cheaper alternatives to accomplish the objective.  A great example would be Fax disrupting Post Office mail and then Fax being disrupted by email.

Unlike the technology disruption of the 80’s and 90’s where the bulk of the disruption in the ECM space came from hardware improvements (faster chips, cheaper memory), the disruption of this century has been around software and the ability to leverage commodity hardware.  The .com boom fueled start-ups from the consumer side as well as Open Source that have added new components to ECM solutions that are better and cheaper and available as open source for all.  Some examples:

• Linux – Early ECM adopters leverage Sun/HP/Microsoft for ECM server components. The majority of clients have shifted from UNIX to Linux.  Microsoft remains active but is losing ground 2 to 1 to Linux based on recent surveys.
• Lucene/Solr – Has disrupted many commercial search vendors. Documentum, for example, used to be based on Verity, switched to FAST and now embraces a combination of Lucene and their XML database.  Alfresco is written completely on Solr.  For many searches, Lucene/Solr has also disrupted the traditional relational database component of ECM vendors like Documentum and Alfresco.

In both of the above examples, cheaper/better tools from the Open Source world are fast replacing proprietary components of an ECM infrastructure.  As we look to the future, we would expect that trend to continue.

Disruptors going forward

As we look past 2015, we are predicting two major technology trends that will be the disruptors moving forward:

• Limitless resources for Storage/Memory/CPU – The trend for both internally purchased as well as cloud storage, memory and CPU is leading to a continued trend for more resources for less cost over the years. Many analysts advise new architectures should be designed for unlimited resources rather than constrained based on current prices.
• Big Data – The consumer vendors (Google, Amazon, Netflix…) are investing in Big Data and related technologies. Rather than look to traditional database vendors (ex: Oracle, Microsoft) and relational database technologies from the 90’s, the Big Data movement is coupling on the limitless resources trend to leverage different technologies, particularly from the open source movement (ex: Hadoop, Mongo).
• Commodity Hardware built for Clustered Scaling – Google has demonstrated that traditional “big hardware” is no longer required. Solutions that can easily scale and adapt will leverage both the limitless resources and Big Data developments.
• ECM vendors – As solutions for reducing the cost components of the ECM architecture emerge, ECM vendors are going to be pushed to embrace these components as well as find ways to improve their ROI.

We would predict that cloud-based offerings, both as hosted ECM solutions or complete software as a service models, are positioned to take advantage of the disruptors listed above.  We wouldn’t group either as a disruptor as much as an alternative to traditional in-house hosting.

Predictions on the Future of ECM

We are predicting that the disruptors of limitless resources along with technology emerging from the Big Data movement will trickle into the ECM architecture to drive the following disruptions.

• Storage Area Network – Rather than the pricy options from EMC or other SAN manufacturers, more and more commodity hardware/open source solutions will replace these costly back-ends.
• Relational Database – Big Data approaches that leverage Lucene/Solr can replace the traditional relational database and it’s associated cost.
• ECM Vendors – As technology moves, the legacy ECM vendors will be hard pressed to embrace technology given their reliance on the relational database and SAN architecture. Innovative vendors will move into this space to challenge with new technology and architectures.

If we had to pick a favorite disrupting technology at this time, TSG is predicting Hadoop as a major contender based on:

• ECM capabilities in Hadoop
• Hadoop capabilities to disrupt the relational database component of ECM

Summary

We are predicting that disruption coming from Big Data as well as efficiencies with storage and database components of an ECM architecture will affect all of the ECM vendors over the next ten years.  We also predict that Hadoop, based on success in replacing traditional database components for Big Data, will be embraced and disrupt both the relational database as well as SAN storage components of ECM architectures.  The ability of legacy ECM vendors to embrace the change will differentiate vendors moving forward.

Let us know your thoughts on our predictions below.