• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
TSB Alfresco Cobrand White tagline

Technology Services Group

  • Home
  • Products
    • Alfresco Enterprise Viewer
    • OpenContent Search
    • OpenContent Case
    • OpenContent Forms
    • OpenMigrate
    • OpenContent Web Services
    • OpenCapture
    • OpenOverlay
  • Solutions
    • Alfresco Content Accelerator for Claims Management
      • Claims Demo Series
    • Alfresco Content Accelerator for Policy & Procedure Management
      • Compliance Demo Series
    • OpenContent Accounts Payable
    • OpenContent Contract Management
    • OpenContent Batch Records
    • OpenContent Government
    • OpenContent Corporate Forms
    • OpenContent Construction Management
    • OpenContent Digital Archive
    • OpenContent Human Resources
    • OpenContent Patient Records
  • Platforms
    • Alfresco Consulting
      • Alfresco Case Study – Canadian Museum of Human Rights
      • Alfresco Case Study – New York Philharmonic
      • Alfresco Case Study – New York Property Insurance Underwriting Association
      • Alfresco Case Study – American Society for Clinical Pathology
      • Alfresco Case Study – American Association of Insurance Services
      • Alfresco Case Study – United Cerebral Palsy
    • HBase
    • DynamoDB
    • OpenText & Documentum Consulting
      • Upgrades – A Well Documented Approach
      • Life Science Solutions
        • Life Sciences Project Sampling
    • Veeva Consulting
    • Ephesoft
    • Workshare
  • Case Studies
    • White Papers
    • 11 Billion Document Migration
    • Learning Zone
    • Digital Asset Collection – Canadian Museum of Human Rights
    • Digital Archive and Retrieval – ASCP
    • Digital Archives – New York Philharmonic
    • Insurance Claim Processing – New York Property Insurance
    • Policy Forms Management with Machine Learning – AAIS
    • Liferay and Alfresco Portal – United Cerebral Palsy of Greater Chicago
  • About
    • Contact Us
  • Blog

FileNet – How to retire in weeks rather than months

You are here: Home / FileNet / FileNet – How to retire in weeks rather than months

April 22, 2019

Many of the TSG customers and prospects on FileNet are struggling with old versions of FileNet that have lived for decades with millions and even billions of documents.  For clients looking to reduce the cost of supporting FileNet as soon as possible, TSG can leverage some innovative solutions, including a rolling migration combined with offline access methods from TSG’s OpenMigrate to allow customers to stop running FileNet considerably earlier than a full migration effort would typically take.  This post will discuss the background and unique migration approach.

Understanding FileNet Image Services

FileNet Image Services was initially created back in the 1990’s for scanning solutions when network bandwidth, database, CPU and memory were all expensive resources.  Solutions, like FileNet, built at that time had some efficiencies created for the limitations of the day that really don’t make that much sense in today’s world and often can make migrations difficult and complex.  Specifically for FileNet, documents are typically stored in TIFF format (think 1990’s fax machine) with a proprietary header information from FileNet.  TSG typically refers to this as FileNet TIFF to not be confused with an open format TIFF.   Each page of a document is stored as a separate file to reduce network traffic as well as to display the first page of a multi-page document as quickly as possible.  Viewing documents required a viewer that is aware of this unique header and page storage.  TSG recommend converting the FileNet TIFF single page documents to multi-page PDF to be more compatible with browser based viewing as well as other PDF capabilities.  FileNet also has a proprietary compressed computer output to laser disc (COLD) format (typically FOB file extension) that also requires a FileNet viewer. 

As we detail in our case study on moving one large client from FileNet to AWS, as well as our previous post on 10 Reasons why there will never be an “Easy Button” for migrations, migrations can be difficult for a number of different reasons. Specifically pressing reasons for FileNet customers typically include:

  • Moving Integrations – Given the age of the system, typically many other systems are connected to FileNet for viewing and need to be moved to a new interface.  The integration move needs to be coordinated with the migration to view all documents in the new system.
  • Moving ingestions – Like integrations, typically there are one to many ingestion jobs that feed FileNet and will need to be moved.  These moves also need to be coordinated with the move to the new system.
  • User Change Management – Given the amount of content on FileNet, releasing a new interface to users timed with when all the content would be available after the migration.

For all of the above reasons and probably more, many FileNet clients tend to just keep FileNet running on old hardware and delay the decision to migrate given the complexity of all of the migration efforts.  For some customers, the decision to delay has resulted in billions of documents in an old and outdated FileNet implementation.

FileNet Image Services migration with OpenMigrate

For our FileNet Migration practice, TSG determined years ago that accessing the FileNet Image Services API was not practical for large scale migrations.  Major reasons included:

  • API Performance – TSG found the performance of the API to not be on par with a high-volume multi-threaded approach.  This could be the API but could also be the typical older hardware supporting FileNet.
  • Ongoing FileNet Usage – Typically clients ask TSG to start migrating documents before the system was formally retired.  Running a migration that would consume resources and might affect the performance for the FileNet users was unacceptable.

For FileNet Image Services Customers, TSG has chosen to leverage our database connector with direct access to the underlying FileNet images to avoid the two scenarios above.  By taking a copy of the database that contains the meta-data and file-pointers to start the migration process, OpenMigrate doesn’t even have to access any of the FileNet components or Filenet even running to run a migration.  Once the migration is complete with the copy of the database, TSG will typically do a small delta migration for any content that has been added to FileNet while the migration was running.

OpenMigrate has both source and target connectors for FileNet P8 that leverage the API. This approach works fine for low-volume migrations. TSG is working on a direct database approach for FileNet P8 for a high volume migrations.

FileNet Migration – Addressing the Ingestion Issue

To address the ties to FileNet ingestions, part of the migration process requires ingestions to begin moving their ingestion to the new system.  We have seen clients be successful with a couple of different approaches.

  • Feed through old system – For this approach, content continues to be fed through FileNet to either be migrated or on demand pushed through to the new system.  Typically TSG can leverage OpenMigrate for this type of ingestion as it has the source adapters for FileNet. This approach does not allow FileNet to be decommissioned until a new feed is built to the new system.  This approach works well when the new integration effort will take longer than the migration effort but does not let FileNet be retired early.
  • Dual feed both systems – For this approach, new content is fed into both systems.  For this approach, the content is available for both users on the old system as well as on the new system.  With this approach the old system can continue to be run while new content and potentially migrated content is ingested into the new system.  This approach works well when content (new/old) can be isolated.  Users can be moved to the new system when the content they need, either through the ingestion or a migration, is moved into the new system.  Once all users and documents are off of the old system, FileNet can be retired.

FileNet Migration – Retiring FileNet in Weeks

Recently, TSG was brainstorming with a client that was looking for a way to allow them to stop paying IBM for FileNet support as soon as possible since they were moving to a new alternative.  For this client, the development work for ingestion into the new FileNet alternative had begun and the client was looking for an easy way to move FileNet content into an archive system.  TSG identified a combination of rolling migration and the unique capability of OpenMigrate to migrate documents from FileNet without requiring FileNet to be actually running.  As we have pointed out here many times before, a rolling migration involves moving some or all users to the new system immediately with the content they require being moved over when requested “on demand”.  This scenario fits case management scenarios where content is broken down into cases that can be retrieved and moved in bulk but would also work for individual documents.  TSG recommends a rolling migration approach for clients with large numbers of documents, users, and integrations.  (see our related post on rolling migrations for Alfresco customers).

The detail of the “turn off FileNet early” approach includes:

  • All the meta-data from FileNet will be migrated to a new repository from the FileNet Database.  This is typically a very fast migration as it is just dumping the metadata without converting any files.  Metadata will include file-pointers to actual pages/documents in the old FileNet format.  TSG can support Alfresco, Documentum, Hadoop, DynamoDB out of the box for the new repository.  Client is considering adding an alternative NoSQL solution.
  • TSG’s OpenMigrate will begin migrating documents from the proprietary TIFF and COLD format to the new system in reverse date order (newest to oldest).  OpenMigrate will be converting both TIFF and COLD into PDF format.  Multiple threads of OpenMigrate will be configured to maximize throughput. 
  • TSG’s OpenContent interface will be installed to view documents and case folders.  Users will be able to search and access meta-data content from the FileNet metadata that has been moved over.  When viewing or accessing documents, if the documents have already been migrated and converted to PDF, the interface will simply show the PDF documents.  If the document(s) have not been migrated, the OCMS Case will initiate an on-demand migration of the document(s) with the FileNet document links.  Leveraging the OpenMigrate threads already running, the interface will place the document(s) in the OpenMigrate queue with priority to process immediately.  The interface will present a timer with a message to the user that the document(s) are being migrated and, when available, will display the document(s).  For previous clients, TSG has gotten 5-10 second performance for on-demand document retrieval.

By leveraging the process above, FileNet Image Services is no longer required as the rolling migration does not leverage anything on the FileNet server.

Summary

By leveraging a rolling migration approach along with TSG’s OpenMigate product, some FileNet Image Services migration scenarios can provide for the early retirement of FileNet itself before all the documents have been converted and migrated to the new platform.  This approach would work for both on-premise as well as cloud migration efforts and allow clients to retire FileNet earlier.

While TSG currently has OpenMigrate connectors for FileNet P8’s APIs, TSG is working on a similar direct method to retrieve content from FileNet P8 without FileNet P8 required to be running..

Download FileNet to AWS Case Study

If you have any thoughts or questions, please enter them below.

Filed Under: FileNet, Migrations, OpenMigrate

Reader Interactions

Trackbacks

  1. Federated Search versus Rolling Migration says:
    May 6, 2019 at 1:36 pm

    […] we have been talking to multiple customers, particularly with FileNet, that have large repositories with hundreds of millions or even billions of documents.  As we […]

    Reply

Leave a Reply Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Primary Sidebar

Search

Related Posts

  • FileNet Migration – Not as hard as you think?
  • FileNet Support – Migrating to mitigate the risk of an unsupportable product
  • FileNet Migrations – Best Practices for Large Migrations
  • File Formats Lessons Learned – Legacy ECM Migrations
  • FileNet COLD Migration – Cracking the proprietary format issue
  • FileNet Migration – Recorded Alfresco/TSG Webinar – 05/29/2019
  • Migrating to Alfresco – Reducing Risk, Stress and Cost with a Rolling Migration
  • FileNet Migration – Best Practices and Client Experience
  • FileNet Migration to Alfresco with OpenMigrate
  • Alfresco Webinar – Benefits of a Rolling Migration versus a Big Bang – October 21st

Recent Posts

  • Alfresco Content Accelerator and Alfresco Enterprise Viewer – Improving User Collaboration Efficiency
  • Alfresco Content Accelerator – Document Notification Distribution Lists
  • Alfresco Webinar – Productivity Anywhere: How modern claim and policy document processing can help the new work-from-home normal succeed
  • Alfresco – Viewing Annotations on Versions
  • Alfresco Content Accelerator – Collaboration Enhancements
stacks-of-paper

11 BILLION DOCUMENT
BENCHMARK
OVERVIEW

Learn how TSG was able to leverage DynamoDB, S3, ElasticSearch & AWS to successfully migrate 11 Billion documents.

Download White Paper

Footer

Search

Contact

22 West Washington St
5th Floor
Chicago, IL 60602

inquiry@tsgrp.com

312.372.7777

Copyright © 2023 · Technology Services Group, Inc. · Log in

This website uses cookies to improve your experience. Please accept this site's cookies, but you can opt-out if you wish. Privacy Policy ACCEPT | Cookie settings
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT