FileNet – Migrating to Amazon S3

We had an interesting call with an insurance client yesterday currently on FileNet but looking for ways to start to move away from FileNet to a more modern solution, preferably in the cloud.  Their existing FileNet implementation was slightly unique in that they didn’t rely on FileNet to store any specific attributes about the document, instead storing all document attributes in their Claim Management system (based on Guidewire) with simple pointers to the FileNet F_DOCNUMBER as the FileNet Image Service unique id.  During our call, we came up with a simple solution to leverage the Amazon Web Services S3 storage to remove FileNet with minimal changes.  This post will describe that approach.

FileNet as an Object Store

Like most image storage systems, FileNet has a two distinct components to their API and content model.  The first component is for application data where attributes, security and data about the document are stored.  Clients will access the application data as part of searches to return a listing of documents to view.

The second component is the storage of the document itself.  Within the Application Data, FileNet will store an F_DOCNUMBER (think if it as an object id) that points to the document.  Growing up in the image world, FileNet built components based on the technology of the time.  For example, older FileNet implementations still store documents as separate page TIFF files (comes from the fax format) with compression and deliver these documents one page at a time.  At the time this approach was developed to reduce network traffic needed to see the first page but required lots of different entries into the database to identify each page.

Our client had only implemented the second FileNet component of the storage of the document itself within FileNet and had stored the F_DOCNUMBER within their own claim management system.  From the claim management system, if a user wanted to view a document, the system would launch the FileNet Viewer with the F_Doc_Number.  The FileNet viewer would view the document as well as allow annotations to be stored in FileNet.  It is important to understand that FileNet doesn’t control security or even know what document type is stored with the F_DOCNUMBER.

Replacing FileNet with AWS Object Store

TSG has been moving customers from FileNet for over 10 years.  Typically we focus on moving both the application data and the object.  One interesting newer feature of Amazon S3 is the ability to store information about the object in an S3 bucket.   See the complete list of Amazon S3 features on the AWS website as well as specifics on Object Tagging (often called MetaData). Leveraging the capability of S3 to provide some basic indexing, the unique components of the FileNet migration include:

1. Replacement of the multi-page Tiff documents to one PDF document.
2. Migration of the FileNet annotations to Adobe XFDF annotations
3. Migration of the new PDF and annotations to Amazon S3 object store with an attribute of the old FileNet F_DOCNUMBER included in the S3 Object Tagging.
4. Movement of the application data in FileNet to the new ECM system (Alfresco or Documentum) and linking the Amazon S3 to the application data via the old FileNet F_DOCNUMBER or the new S3 object id.

In the case of our specific client, we quickly realized that we didn’t really need to do number 4 from above immediately if the client just wanted to swap out the FileNet object store with an Amazon object store.  In discussions with the client, they are considering moving away from their current claim management system and would look to move document specific attributes while gaining other benefits of a modern ECM system at a later point.

Some specifics on the new access would include:

• The migration would store the FileNet F_DOCNUMBER as an attribute on S3 for both the document as well as the annotations. AWS S3 allows for up to 2KB of attributes per object.
• The claim system would launch OpenAnnotate to view the object from S3 rather than the FileNet viewer. Annotations would be stored in S3 with Object ID, Annotation as well as User information.
• We could consider a “rolling migration” where documents are migrated to S3 from FileNet on demand when accessed.

Summary

For FileNet or other legacy ECM implementations that are just treating the ECM system as a way to store files, Amazon S3 offers basic object storage with attributes along with the benefits of modern document formats and viewers at a cost effective price in the cloud.  With an innovative approach, some clients may be able to move quickly to cloud based storage and move away from an expensive on-premise FileNet installation.