• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
TSB Alfresco Cobrand White tagline

Technology Services Group

  • Home
  • Products
    • Alfresco Enterprise Viewer
    • OpenContent Search
    • OpenContent Case
    • OpenContent Forms
    • OpenMigrate
    • OpenContent Web Services
    • OpenCapture
    • OpenOverlay
  • Solutions
    • Alfresco Content Accelerator for Claims Management
      • Claims Demo Series
    • Alfresco Content Accelerator for Policy & Procedure Management
      • Compliance Demo Series
    • OpenContent Accounts Payable
    • OpenContent Contract Management
    • OpenContent Batch Records
    • OpenContent Government
    • OpenContent Corporate Forms
    • OpenContent Construction Management
    • OpenContent Digital Archive
    • OpenContent Human Resources
    • OpenContent Patient Records
  • Platforms
    • Alfresco Consulting
      • Alfresco Case Study – Canadian Museum of Human Rights
      • Alfresco Case Study – New York Philharmonic
      • Alfresco Case Study – New York Property Insurance Underwriting Association
      • Alfresco Case Study – American Society for Clinical Pathology
      • Alfresco Case Study – American Association of Insurance Services
      • Alfresco Case Study – United Cerebral Palsy
    • HBase
    • DynamoDB
    • OpenText & Documentum Consulting
      • Upgrades – A Well Documented Approach
      • Life Science Solutions
        • Life Sciences Project Sampling
    • Veeva Consulting
    • Ephesoft
    • Workshare
  • Case Studies
    • White Papers
    • 11 Billion Document Migration
    • Learning Zone
    • Digital Asset Collection – Canadian Museum of Human Rights
    • Digital Archive and Retrieval – ASCP
    • Digital Archives – New York Philharmonic
    • Insurance Claim Processing – New York Property Insurance
    • Policy Forms Management with Machine Learning – AAIS
    • Liferay and Alfresco Portal – United Cerebral Palsy of Greater Chicago
  • About
    • Contact Us
  • Blog

Federated Content Management – Enterprise Search with a new moniker?

You are here: Home / Alfresco / Federated Content Management – Enterprise Search with a new moniker?

November 14, 2019

One troubling point we continue to see in our industry space, whether it be content management providers like Nuxeo, ASG or even analysts like Gartner, is a continued push around the benefits of a federation search, often as an alternative to migration.  This post will explore federated content management with a comparison to a very similar approach marketed as Enterprise Search from almost 20 years ago, a technology that failed to produce value. 

Enterprise Search – What was the promise?

Back in the early 2000’s, Enterprise Search held a promise for content management very similar to what is being promised from Federated Search now:

  • Knowledge workers access multiple content management systems.  A tool that could streamline knowledge worker access and activity would improve worker efficiency.  Typical quotes include “knowledge workers spend X% of their time searching for information resulting in a cost of Y.”
  • Migrating from a legacy content management system is difficult.  A tool that could provide access to the old system while also providing access to the new system would reduce the need to migrate everything from the old system to the new system.

Enterprise Search promised the “single pane of glass” to view all content in an enterprise.  Stats were throw around about just how much time Knowledge workers waste searching for content.  Enterprise Search was viewed as the way to reduce the time looking for content in multiple systems. 

At the time, TSG was an Autonomy partner with multiple engagements for Autonomy Enterprise Search implementation.   

Enterprise Search – What were the problems?

For our clients, as well as other Autonomy customers, some of the struggles they encountered included:

  • Adapters and Support – For Enterprise Search, Autonomy needed adapters to connect to the existing (often legacy) systems to be able to extract and index the content.  Typically, TSG would build or modify adapters as part of our implementation services.  As Autonomy (or TSG) didn’t maintain our own copies of these legacy implementations, any support would require access to the client’s system.  For issues, TSG was often “on our own” when it came to getting support from the legacy system vendor as typical legacy systems didn’t always have easy support connections.
  • Security – Enterprise Search would index the content in the search engine for quick and efficient combined search.  Pushing different content repositories content required also providing a vehicle that would protect and replicate security to the search index to protect secure documents. Security added complexity and required a re-index if any security changed.
  • Updating Content – While finding content makes sense, what happens when the user wants to do something with the content?  Activities could include annotation or combining with other content or linking for a specific use case.  Enterprise search suffered in that it could only view the document and would have to access the source application to work with the document.
  • Licensing – Depending on the approach, users would require licenses for both the Enterprise Search tool as well as the legacy repositories.

Federated Search – What are the differences and similarities?

Federated Search is different than Enterprise Search in that the approach leverages the existing content repositories search (and security) rather than building a new index in the content repository tool.  Where the Enterprise Search approach looks like:

  1. Backend – Search tool access content repository, build new search index for all repositories.
  2. User accesses new search index.
  3. User views document from old repository.

Federated Search looks like:

  1. Search current repository and legacy content repository leveraging legacy APIs
  2. Combine search results.
  3. User views documents from combined interface.

The major difference and advantage for federated search is that a new search index is not created.  This allows security to be enforced as the federated search is leveraging the security of the legacy content repository.  Issues with Federated Search include:

  • Combining Search is not that simple – As different repositories have different content models and values, searches done in one repository are typically different in another requiring an understanding of both (or more) repositories at a detailed level.  Federated tools rarely combine the searches, just allow a search against one repository or the other.
  • Combining Search can be inefficient – Enterprise search offered one efficient index for searching.  Federated search requires access to multiple repositories with the search returning at the pace of the slowest repository.  In looking under the covers at one of the main federated search vendors, they are really migrating the meta-data to do one search and just leaving the file binaries in place in the old system. This approach requires the duplicating of the meta-data as well as constant syncing between the repositories for any changes.
  • Adapters and Support – Like Enterprise Search, adapters have to be written (and supported) for the legacy platform.  Supporting the integrations is very difficult given that 1)  the product is legacy by definition, 2) the product is often competitive to the new content repository, and 3) Older systems are often less stable and harder to support than new systems.
  • Updating Content – Similar to Enterprise Search, what happens when the user wants to do something with the content?  Activities could include annotation or combining with other content or linking for a specific use case.  Does this happen in the old repository?  Can the interface handle that update?
  • Licensing – In this case, the license for both the old and new content repositories would need to be maintained with users having valid licenses on both systems. Older system would not be able to be decommissioned.

Of the above issues, we would say that the adapters and support from a software product perspective is the most difficult and probably looks more like a consulting engagement where the consultant has access to the legacy environments vs. software only approach.  Any support call would require the federated software firm to:

  • Access the customer’s environment to debug the issue as the company probably doesn’t have their own instance of the legacy repository.
  • Understand both the new repository as well as legacy repository because the legacy repository would not be providing support.

“While Enterprise Search promised value, even my clients that were able to get over the security, support and development issues of the adapters found that the final solution just wasn’t being used”, said Alan Pelz Sharpe, long time ECM/CPS analyst. “The business case for consolidating search and view of content just didn’t add enough value versus accessing the source systems themselves.”

Federated Search with a Rolling Migration

TSG approaches Federated Search as a means to a migration rather than in place of one.  From our post on Federated Search versus Rolling Migration, see the differences in the table below:

Federated Search Federated Search with Rolling Migration
Search Retrieves the
Legacy document
Search retrieves the legacy
document
Transform the document
for viewing
Transform the document for
viewing
View the document Store the transformed document
in new repository
  View the document from new repository

In the above case, the 2nd time the document was requested, the document would be immediately viewed from the new repository. 

In the above scenario, the benefit of Federated Search is quicker viewing of the document the first time it is accessed and those savings might just be sub-second depending on how the new system is set up. 

Benefits of the Rolling Migration approach include:

  • Quicker viewing of documents the second and subsequent views.
  • Ability to View/Annotate/Combine and any other document manipulation function in the new system.
  • Ability to not have to transform the document the second time and beyond, a non-trival effort when working with examples like FileNet Image Services where the documents could be proprietary COLD or single page TIFF that have to be combined to PDF along with potential Daeja annotations.

When combined with a migration effort, the differences are also distinct.

  • The rolling migration will always look into the legacy system to determine if something is different between the new system and legacy system and move it if required.
  • The federated search will have to rely on some other method to know when to search the legacy system and the new system.  Content dropped into the old system by an old legacy integration could prove tricky.

To address the specific points brought up above:

  • Combining Search is not that simple – TSG will typically handle the search as part of a migration effort where we will build and test the search and migration for the client’s specific scenario providing real-time support throughout the migration.
  • Adapters and Support – As part of a consulting agreement, TSG brings previous adapters but tailor them for a specific client and their environment and versions.
  • Updating Content – As content is brought over to the new system, no issue exists with updating content.
  • Licensing – With a migration approach, eventually the old system can be retired.  For some approaches, like our FileNet Migration, we can access the content without requiring the system to be running or support to be paid. This approach can lead to old systems being decommissioned sooner.

Client Experiences – Federated Search and Rolling Migration

One thing we continue to struggle to understand about Federated Search is successful customers and case studies.  Most blogs talk about what it can do rather than success of it actually doing it similar to the Enterprise Search promises.  TSG did a comparison of three large customers, two that did rolling migrations and one that did more of a federated-bulk migration.  Some interesting points:

  • One Rolling Migration client was able to start with only 600,000 documents in the repository and pilot/gradually roll out from 1,000 users to 19,000 users and 350,000,000 documents.
  • The Bulk Migration client was not able to pilot, attempted a big bang migration approach after a 11 month migration and ended up falling back to their legacy content repository.
  • One Rolling Migration client still uses old feeds into an older repository (custom – not a vendor solution) rather than moving feeds to the new repository.  Federated Search approach would not work for this client as the old custom repository has no APIs and required database look-up to support the rolling migration.

Summary

A rolling migration can be a game changer when migrating from multiple legacy systems to a new system, either on premise or in the cloud by reducing the need to move all users, content and integrations before the new system can be used and piloted.  TSG has assisted multiple large clients and customers to successfully move to new systems by leveraging a rolling migration approach.  Federated Search can play a role in accessing content from legacy systems but these applications will struggle with document actions like annotation or other document manipulation as well as licensing and other support requirements.

Let us know your thoughts below.

Filed Under: Alfresco, Documentum, ECM Landscape, Migrations

Reader Interactions

Trackbacks

  1. Federated Content Aggregation – Comparison to Federated Search and Rolling Migration — Technology Services Group says:
    March 13, 2020 at 8:25 am

    […] in 2019, we posted thoughts on Federated Content Management – Enterprise Search with a New Moniker?  At the time we posted comparisons between Enterprise Search, Federated Search and a Rolling […]

    Reply

Leave a Reply Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Primary Sidebar

Search

Related Posts

  • ECM 2.0 – One-Step vs. Two-Step Migrations
  • ECM Roadmap – Thoughts on Planning for the Future
  • Documentum, SharePoint, Alfresco – Document Control for Life Sciences
  • Alfresco Consulting – Documentum Disruptor #2
  • Documentum Migration to Alfresco – Summary of Series Postings
  • Documentum Migration to Alfresco – Part 1
  • Third Annual TSG Client Briefing – June 3rd – 2010
  • ECM 2.0 – Can you build it yourself?
  • The Deep Analysis Podcast – The 11 Billion File Benchmark
  • ECM Sales Myths for 2019

Recent Posts

  • Alfresco Content Accelerator and Alfresco Enterprise Viewer – Improving User Collaboration Efficiency
  • Alfresco Content Accelerator – Document Notification Distribution Lists
  • Alfresco Webinar – Productivity Anywhere: How modern claim and policy document processing can help the new work-from-home normal succeed
  • Alfresco – Viewing Annotations on Versions
  • Alfresco Content Accelerator – Collaboration Enhancements
stacks-of-paper

11 BILLION DOCUMENT
BENCHMARK
OVERVIEW

Learn how TSG was able to leverage DynamoDB, S3, ElasticSearch & AWS to successfully migrate 11 Billion documents.

Download White Paper

Footer

Search

Contact

22 West Washington St
5th Floor
Chicago, IL 60602

inquiry@tsgrp.com

312.372.7777

Copyright © 2023 · Technology Services Group, Inc. · Log in

This website uses cookies to improve your experience. Please accept this site's cookies, but you can opt-out if you wish. Privacy Policy ACCEPT | Cookie settings
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT