Federated Search versus Rolling Migration

In moving our legacy repositories customers to new modern solutions, one of the major recommendations to reduce cost, risk and stress is a rolling migration.  In a discussion with a client last week, there was some confusion in mixing a rolling migration with a federated search.  This post will discuss the differences and how a rolling migration is more practical for moving and retiring legacy repositories.

Rolling Migration – Why is it a game changer?

Recently we have been talking to multiple customers, particularly with FileNet, that have large repositories with hundreds of millions or even billions of documents.  As we have discussed many times before, there is no easy button for migrations, particularly when systems have been around so long and have so much content, users and integrations.  Major reasons the migration can be so difficult include:

• Integrations – often overlooked, the feeds into the old system can be the most difficult to move to the new system as the resources that built the feeds are probably no longer with the company or have moved on to new roles.  There is also a timing issue in regards to when to stop feeding the old system and feed the new system.
• Volume of Content – migrations with millions of documents just take time.  Particularly when changing format of the document from a FileNet TIFF to a more standard PDF. 
• User change management – when systems have billions of documents, they often have thousands of users.  Coordinating the move with training and the move of content and integrations can be very complex and difficult.  Too many users thrown on the a new system on Day 1 can result in user perception issues for any system or training issues.

TSG recommends a rolling migration to allow integrations, content migration and user change management to all proceed in separate distinct threads to allow all three to gradually “roll” onto the new system.  For an example, let’s take a FileNet Image Services migration for a large repository with insurance claims moving to Alfresco.

1. TSG first would configure OpenMigrate and the OpenContent Case Interface to connect to FileNet.
2. On day one, when a user requested a claim folder, OpenMigrate would access FileNet and move that claim to Alfresco “on demand”. If the claim doesn’t yet exist in Alfresco, the claim folder is created and the FileNet contents are migrated into Alfresco. This will allow viewing, annotating, combining documents from the OpenContent Case and OpenAnnotate interface.  For FileNet Image Service documents, moving would include transforming the documents to multi-page PDFs from single page TIFFs with Daeja annotations.  This migration could happen to Alfresco on premise or also into Alfresco in the cloud.
3. On subsequent requests for the same claim, OpenMigrate would always look in FileNet to verify that nothing new has been added to FileNet for the claim and migrate over if necessary.

In the manner above, different threads can be set up for:

• Integrations don’t need to be updated or moved immediately.  Strategies can be developed to gradually move the integrations or even develop strategies to move integrations to feed both the old and new systems for a time.
• Content doesn’t need to be moved immediately.  Most clients will start a parallel migration effort of newest to latest but, as we mentioned in our post on rolling migrations (add link), the migration effort can coincide with the rolling of on-demand documents. 
• Users can be gradually rolled on to the new system.  Pilot users can begin using the new system and other users can be added over time.  Users can be rolled on by department or other group coordinated with training.  System tuning and stability can be coordinated with more documents, users and integrations being added to the system.

Federated Search – Can it play a role?

Some customers have looked to see if having search access both the new and old system could play a role.  Compare the two scenarios below:

Federated Search	Rolling Migration
Search Retrieves the Legacy document	Search retrieves the legacy document
Transform the document for viewing	Transform the document for viewing
View the document	Store the transformed document in Alfresco
	View the document from Alfresco

In the above scenario, the benefit of Federated Search is quicker viewing of the document the first time it is accessed and those savings might just be sub-second depending on how the new Alfresco system is set up.  Benefits of the Rolling Migration approach include:

• Quicker viewing of documents the second and subsequent views.
• Ability to View/Annotate/Combine and any other document manipulation function in the new Alfresco system.
• Ability to not have to transform the document the second time and beyond, a non-trival effort when working with FileNet Image Services where the documents are single page TIFF that have to be combined to PDF along with potential Daeja annotations.

When combined with a migration effort, the differences are also distinct. 

• The rolling migration will always look into the legacy system to determine if something is different between the new Alfresco system and legacy system and move it if required.
• The federated search will have to rely on some other method to know when to search the legacy system and the new Alfresco system.  Content dropped into the old system by an old legacy integration could prove tricky.

Summary

A rolling migration can be a game changer when migrating from multiple legacy systems to a new system, either on premise or in the cloud by reducing the need to move all users, content and integrations before the new system can be used and piloted.  TSG has assisted multiple large clients and customers successfully move to new systems by leveraging a rolling migration approach.  Federated Search can play a role in accessing content from legacy systems but will struggle with document actions like annotation or other document manipulation.