ECM Migrations – 10 Reasons there will never be an “Easy Button” for migrations

Just about every couple of years we will see a campaign from a software vendor or ECM integrator claiming that their new product or approach has made ECM migrations easier.  Typically the marketing campaign will focus on a new sales initiative looking to more easily move clients from their old ECM platforms and software to vendors ECM platforms and software and probably the cloud.  TSG is the leader in migrating clients to new ECM systems with 15+ years of experience.  This post will present our thoughts on why there will never be an “easy” button when it comes to most legacy ECM migrations.

Reason #1 – If it ain’t broke… finding a compelling reason to move

In 2018, most of the legacy ECM systems that are around have been in or near production for 10+ years.  Typically the systems have been tweaked and enhanced over that time.  Coming up with an easy button for the migration involves coming up with an easy justification for spending the money moving to a new system.  Migrating to a new ECM system involves these steps:

• Install new repository
• Configure new object model, new interface
• Train users
• Replace integrations
• Migrate content

All of the above involve cost, time, and money.  As reported by Doculabs late last year, Doculabs was even surprised with just how sticky Documentum turned out to be in their own survey of Documentum users.  Making it easy for clients to move also means making the justification easy.  Given a risk-adverse environment, it is often just too easy to kick the can down the road and delay any migration until next year.

Reason #2 – User Requirements and User Acceptance

When TSG is asked, “What is the biggest risk of moving to the new system?” our typical answer has always been user acceptance.  Legacy ECM systems that have been in production 10+ years typically fulfill a variety of different user requirements and have customizations that have evolved with the system over time.  Functionality is often introduced as a major feature of the old system that doesn’t easily translate to the new system.  For example, early Documentum had a concept of “Virtual Documents” that was built for publishing but was often used by developers for a variety of non-publishing solutions.  Users might see their requirement for virtual documents as a business requirement that is very difficult to fulfill in the new system because virtual documents was a proprietary approach by Documentum that has been discontinued.

Documenting and including all of the different user requirements from the old system can be a difficult and time consuming process.  Mapping the old requirements to the new system and interface will require detailed discussion, and often compromise, as new interfaces usually handle requirements in different ways.  Requirements that do not add enough business value to justify their cost and risk of development in the new system need to be considered as well.  IT needs to keep in mind that introducing a new system represents a substantial change.  Users will, and do, push hard to include all of the existing requirements plus new capabilities.

Reason #3 – Migration Exceptions & Bad Data

Old systems typically have older documents.  In running our large migrations for clients we have found:

• Bad File Pointers – content missing in source system
• Duplicate Data
• Corrupt Content
• Invalid File Names
• Missing Metadata
• Unknown File Types

The above, along with a host of other unknown issues, could stall or substantially slow down any migration.  In a typical large migration (millions of documents), it is very difficult to identify all the potential error conditions without somehow testing or scanning the repository before the full production migration.  For example, as we pointed out earlier this year, Alfresco’s Bulk Import does not have an error logging capability or the ability to continue the migration if an error is encountered.  Clients leveraging the Bulk Import for migration efforts must manually identify and correct errors.

Reason #4 – Integrations and Data Lookups

One reasons old ECM systems tend to be sticky is that, over time, they have been integrated with a variety of different business processes.  In order to “lift and shift” to a new system, all of the integrations needed to be moved as well.  Oftentimes the integrations were developed by resources that are no longer with the company and were poorly documented and maintained.  One TSG client had over 60 integrations to their custom ECM system that had been developed over 20+ years.

Coordinating the movement of the integrations as well as the migration of data, new interface configuration, and training of users can be a massive effort.  TSG will often look to mitigate the risk of this effort with a hybrid migration approach to gradually roll on content and users where both the old and new system will be running concurrently.  While this process reduces risk, it requires both systems to be available for some time while the integrations, content and users are gradually moved to the new system, reducing the benefits of retiring the old system.

Reason #5 – Migration Resources

To conduct a successful migration, resources need to be familiar with

• The old ECM system and data
• The new ECM system and data
• The migration tool

Too often, clients will look to leverage their current old system resources to both develop the new ECM system, as well as understand the migration tool.  Typically these are resources that are still required to maintain the old ECM system.  Internal resources often lack the experience in both the new system and migration tool to fully understand the complexity of a large migration.

Even if clients look to hire out consultants for the new ECM system and migration tool, those resources need to take the time to understand the old ECM system and the specific client implementation.  Best practices are a combined team of internal resources and external consultants with a focus on planning and testing the migration.

Reason #6 – Migration Content Updates/Changes

When moving content from one repository to a new repository, often the content needs to be updated.  For example, many old FileNet systems stored multiple page documents as single page TIFFs to improve viewing performance given old network speeds.  Typically a migration will involve moving the single page TIFFs to one multi-page PDF along with annotations, and is not as simple as just moving the files.  Other format changes and updates need to be considered along with possible renditions of documents/files if new features are going to be introduced.  Typically TSG will recommend converting or creating PDF renditions for all documents that can be printed (ex: Word, Excel, TIF…).  With our Video Annotation capabilities, we would also recommend converting all videos to MP4.  Providing consistent renditions allows for a better browser viewing experience.

In discussing PDF renditions, for large migrations, typically clients might decide to rely on the new system to render new PDF renditions for the content being migrated.  While we have been able to scale our content migrations to 250-400 documents a second, renditioning can be much longer depending on the file type (Word for example).  Migration planning for renditioning needs to be calculated as well and could involved migrating the renditions rather than generating with PDF renditioning services.

Reason #7 – Change Management for the New System

Introducing a new ECM system will typically include a new interface and training for the users.  Change management is critical for successful user adoption.  Innovative clients have used focus groups and promotional campaigns to introduce the new system and celebrate the improvements to successfully avoid any issues with user acceptance and adoption.  IT needs to keep in mind that change for document workers can often be seen as a burden rather than an improvement.  Celebrating the new features and capabilities can help users to see how their situation is improved rather than complicated.  Developing a migration and change management approach that eases users into the new system is a critical component of user acceptance and adoption.

Reason #8 – Migration AND Testing

Too often migration efforts will short-cut testing of the migration approach as well as the new ECM system.  TSG will often recommend parallel system efforts to provide users with an environment to fully test out the new system with the migrated content.  If content errors are found, the news system can be wiped and the migration (or parts of it) refreshed with a new migration.  Testing should include both a sampling as well as a test scenarios for a wide range of different documents and content types.

Reason #9 – Differences in ECM Capabilities

Too often, migrations are simplified with the thought of just moving the files and meta-data.  While most modern ECM systems can “check the box” for having all the different functions, most ECM systems implement those capabilities in different ways.  Different components that might be very different between ECM systems include:

• Renditions
• Overlays and Annotations
• Security
• Versioning
• Object Model
• Lifecycle
• Workflow

Moving the files also means moving the different components around the files.  Stopping and starting workflows, applying custom security, versioning, and lifecycle rules all need to be considered for complex business automation document management systems.  ECM capabilities need to be carefully considered as not all migration tools can even process all the different components.  For example, as we reported earlier this year, tools like Alfresco’s Bulk Import does not have the capability to migrate all of the items listed above.

Reason #10 – Audit Trail and Compliance

When moving the documents, the compliance components of the new system need to be considered as well.  One common example is the document create date.  When created in the old system, the document create date was system generated.  When moving the document to the new system, the system might automatically generate the create date as the date of migration.  Typically TSG will recommend moving away from relying on system generated values to make migration easier and store and maintain create date in a non-system generated field maintained by the application.  For Documentum users, we have be writing for years about avoiding using r_object_id as part of their implementation.

The migration effort needs to include moving all of the audit trail, electronic signatures, and other components to maintain the legal compliance of the system.

Summary

Migrating from an old ECM system to a new ECM system, regardless of technology or vendor, will involve building a detail justification and understanding of the nuances of both the old and new systems and include a detailed understanding of:

• The compelling reason and justification for the move
• User Requirements and Acceptance
• Migration Exceptions and Bad Data
• Integrations and Data Lookups
• Migration Resources
• Migration Content and Updates
• Change Management
• Migration and Testing
• Differences in ECM capabilities
• Audit Trail and Compliance

Successful migrations leverage experienced teams, resources, and products to reduce the risk, cost, and timeframe of what can be a very complex, and far from easy, effort.

9-10-2018 Update – Check out Alan Pelz-Sharpe excellent podcast on the issues with migration