• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
TSB Alfresco Cobrand White tagline

Technology Services Group

  • Home
  • Products
    • Alfresco Enterprise Viewer
    • OpenContent Search
    • OpenContent Case
    • OpenContent Forms
    • OpenMigrate
    • OpenContent Web Services
    • OpenCapture
    • OpenOverlay
  • Solutions
    • Alfresco Content Accelerator for Claims Management
      • Claims Demo Series
    • Alfresco Content Accelerator for Policy & Procedure Management
      • Compliance Demo Series
    • OpenContent Accounts Payable
    • OpenContent Contract Management
    • OpenContent Batch Records
    • OpenContent Government
    • OpenContent Corporate Forms
    • OpenContent Construction Management
    • OpenContent Digital Archive
    • OpenContent Human Resources
    • OpenContent Patient Records
  • Platforms
    • Alfresco Consulting
      • Alfresco Case Study – Canadian Museum of Human Rights
      • Alfresco Case Study – New York Philharmonic
      • Alfresco Case Study – New York Property Insurance Underwriting Association
      • Alfresco Case Study – American Society for Clinical Pathology
      • Alfresco Case Study – American Association of Insurance Services
      • Alfresco Case Study – United Cerebral Palsy
    • HBase
    • DynamoDB
    • OpenText & Documentum Consulting
      • Upgrades – A Well Documented Approach
      • Life Science Solutions
        • Life Sciences Project Sampling
    • Veeva Consulting
    • Ephesoft
    • Workshare
  • Case Studies
    • White Papers
    • 11 Billion Document Migration
    • Learning Zone
    • Digital Asset Collection – Canadian Museum of Human Rights
    • Digital Archive and Retrieval – ASCP
    • Digital Archives – New York Philharmonic
    • Insurance Claim Processing – New York Property Insurance
    • Policy Forms Management with Machine Learning – AAIS
    • Liferay and Alfresco Portal – United Cerebral Palsy of Greater Chicago
  • About
    • Contact Us
  • Blog

FileNet COLD Migration – Cracking the proprietary format issue

You are here: Home / FileNet / FileNet COLD Migration – Cracking the proprietary format issue

July 10, 2019

One of the most difficult things about migrating off of FileNet happens for those clients that have relied on leveraging the FileNet COLD format for storage of computer generated documents from internal systems.  Built in the 1990’s in a time when network bandwidth and storage conservation was paramount, the format is unlike any others, badly documented online and very difficult to easily convert into non-proprietary, standard formats like PDF.  This post will present how TSG was able to crack the format issue to build our own FileNet COLD to PDF conversion utility as part of our OpenMigrate product.

FileNet COLD background

COLD stands for “Computer Output to Laser Disc” and was leading edge technology when the cost of spinning magnetic disc was prohibitive for storage of documents or images.  Systems like FileNet evolved to store all documents on optical platters complete with jukeboxes for moving optical platters to drives when the images were requested.  When a jukebox would fill, the software would provide for ways to manage platters offline where, when an image was requested, the user would wait until an operator would feed the platter into the jukebox.

Components of the COLD process involved a routine to call by the client’s output system to create and store the document in the COLD format as well as a viewer that could retrieve and view the COLD format.  The FileNet COLD format includes:

  • Outer Header
    • Compression and document size Information
      • COLD files’ content itself can either be compressed or uncompressed.
      • Compressed content needs to be decompressed before the inner content  header can be parsed.
      • Potentially Background Image (template) Id
  • Content Header – Print Information
    • Background Image (template) Id
      • The template/background for document, as the COLD file itself only includes the text that’s actually different from document to document.
      • Finding template ID: https://www-01.ibm.com/support/docview.wss?uid=swg21957983
    • Display Resolution, scale & orientation
    • Font Information
  • Content
    • Text Data

Most of the high-level information about the format could be found here : ftp://public.dhe.ibm.com/software/data/cm/filenet/docs/isdoc/412x/COLD.pdf

As seen in the above link however, the specific details about the COLD Format were harder to come by, particularly for the Decompression and Background Image Id, and most methods required tools from IBM.

Decrypting the FileNet COLD Format

There are multiple ways to turn the COLD format into PDF.  Two standard ways include:

  • FileNet Viewer – Daeja Viewer  Automation – One method, similar to other transformations to PDF (like Microsoft Office) involves bringing up the FileNet or Daeja viewer in an automating fashion and initiating the print to PDF function.  For small volumes, this method works well but requires the infrastructure (multiple machines – automated scripts) but typically fails for large system migrations with millions of documents to convert.
  • FileNet API  – Some automation can be done with the FileNet API but also requires the infrastructure and scripts to one by one convert the files. 
  • Daeja Viewer API – Similar to the FileNet API, but can be done using only the viewing application itself.  Requires additional scripts to push files into the viewer, but allows the use of a more light-weight application’s API. 

For the first two methods, the approach relies on the access to the FileNet API and old COLD retrieval systems.  TSG has found a couple of issues with either approach including:

  • Performance of the FileNet API/Infrastructure – typically the old FileNet system, built for a previous time, can perform slowly for converting documents.  As we are typically recommending a rolling migration for large FileNet systems, in the majority of cases, the slow performance will not be acceptable.
  • Access to FileNet during migrations – another downside to the approach is requiring access to FileNet during the migration.  For clients wanting to retire FileNet early or for parallel migrations where content is being migrated while FileNet is still being used and contention could slow down all the production users, this approach is also unacceptable.

FileNet COLD – Developing a better way

As part of the OpenMigrate product set, TSG has developed a Java based adapter for FileNet COLD transformation.  The COLD adapter will take FileNet COLD documents and convert them to PDF.  Components of the utility include:

  • Parser for the outer COLD header
  • Decompression of the COLD Content as needed
  • Decoding the inner Content Header for display parameters.
  • Retrieval of the background image based on header info.
  • Conversion of COLD File’s page(s) to a single PDF, using decompressed page’s data, display settings and background image.

By converting the document with just a Java component, the COLD adapter has the following benefits over other traditional conversion approaches.

  • Native File Access – The COLD adapter does not require FileNet to access and convert COLD files.  For both scenarios where FileNet is still running and contention might cause user performance issues or where FileNet is to be retired as part of a rolling migration, native file access allows for fulfilling both of these scenarios.
  • Performance – Compared to traditional “launch a viewer and print to PDF”, by working at the file level, the TSG transformation utility can transform files much quicker to better support rolling migration and bulk migrations.
  • Multi-Threaded Infrastructure – Rather than requiring multiple client machines or other complicated infrastructure, the adapter can take advantage of OpenMigrate’s multi-threaded capabilities to quickly process multiple COLD files at the same time from a single machine.

See below for a short video

Summary

One of the most difficult things about migrating off of FileNet is the proprietary FileNet COLD format for storage of computer generated documents.  Traditional approaches that rely on the FileNet API or FileNet viewers are good for small volumes of documents but fail in large migrations or rolling migration scenarios.  TSG has developed a native file adaptor as part of our OpenMigrate product set that provides better performance as well as superior support for a variety of formats. 

Contact us to learn more and let us know your thoughts below.

Filed Under: FileNet, Migrations, OpenMigrate Tagged With: COLD Format, FileNet, Migration

Reader Interactions

Trackbacks

  1. FileNet Migration – Not as hard as you think? — Technology Services Group says:
    January 23, 2020 at 2:49 pm

    […] OpenMigrate offers code and methods to move FileNet TIFF and COLD to PDF.  For COLD, we discussed our approach in July of 2019 how to simplify the content […]

    Reply
  2. 11 Billion Documents, 12 Months Later - Thoughts and best practices 1 year after our industry leading document benchmark. — Technology Services Group says:
    May 21, 2020 at 8:27 am

    […] one takes time, and the usually the older the system, the longer the conversion will take. Converting the COLD format including parsing and decoding two separate headers, decompression, backg… Understanding your legacy platform, and its limitations, leads to a better understanding of the […]

    Reply
  3. CMOD Migration to a Modern Environment — Technology Services Group says:
    July 14, 2020 at 7:43 am

    […] One unique difference between AFP and other content types is that, to save storage costs, AFP contains the data needed for a print but requires a template to convert into a viewable image.  For example, for a credit card statement, the AFP file might just have the detail about the customer (name, address, charges) but the template would contain the detail about the document type, formats and vendor information.  In this manner, the actual data file can be more compact, providing performance and storage efficiencies.  TSG found the same capability with another proprietary IBM format, FileNet COLD. (See our FileNet Cold conversion approach). […]

    Reply

Leave a Reply Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Primary Sidebar

Search

Related Posts

  • Migrating From Documentum with OpenMigrate: Best Practices
  • Migrating to Alfresco – Reducing Risk, Stress and Cost with a Rolling Migration
  • FileNet Migration – Not as hard as you think?
  • FileNet Support – Migrating to mitigate the risk of an unsupportable product
  • FileNet Migrations – Best Practices for Large Migrations
  • File Formats Lessons Learned – Legacy ECM Migrations
  • FileNet Migration – Recorded Alfresco/TSG Webinar – 05/29/2019
  • FileNet – How to retire in weeks rather than months
  • FileNet Migration – Best Practices and Client Experience
  • FileNet Migration to Alfresco with OpenMigrate

Recent Posts

  • Alfresco Content Accelerator and Alfresco Enterprise Viewer – Improving User Collaboration Efficiency
  • Alfresco Content Accelerator – Document Notification Distribution Lists
  • Alfresco Webinar – Productivity Anywhere: How modern claim and policy document processing can help the new work-from-home normal succeed
  • Alfresco – Viewing Annotations on Versions
  • Alfresco Content Accelerator – Collaboration Enhancements
stacks-of-paper

11 BILLION DOCUMENT
BENCHMARK
OVERVIEW

Learn how TSG was able to leverage DynamoDB, S3, ElasticSearch & AWS to successfully migrate 11 Billion documents.

Download White Paper

Footer

Search

Contact

22 West Washington St
5th Floor
Chicago, IL 60602

inquiry@tsgrp.com

312.372.7777

Copyright © 2023 · Technology Services Group, Inc. · Log in

This website uses cookies to improve your experience. Please accept this site's cookies, but you can opt-out if you wish. Privacy Policy ACCEPT | Cookie settings
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT