• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
TSB Alfresco Cobrand White tagline

Technology Services Group

  • Home
  • Products
    • Alfresco Enterprise Viewer
    • OpenContent Search
    • OpenContent Case
    • OpenContent Forms
    • OpenMigrate
    • OpenContent Web Services
    • OpenCapture
    • OpenOverlay
  • Solutions
    • Alfresco Content Accelerator for Claims Management
      • Claims Demo Series
    • Alfresco Content Accelerator for Policy & Procedure Management
      • Compliance Demo Series
    • OpenContent Accounts Payable
    • OpenContent Contract Management
    • OpenContent Batch Records
    • OpenContent Government
    • OpenContent Corporate Forms
    • OpenContent Construction Management
    • OpenContent Digital Archive
    • OpenContent Human Resources
    • OpenContent Patient Records
  • Platforms
    • Alfresco Consulting
      • Alfresco Case Study – Canadian Museum of Human Rights
      • Alfresco Case Study – New York Philharmonic
      • Alfresco Case Study – New York Property Insurance Underwriting Association
      • Alfresco Case Study – American Society for Clinical Pathology
      • Alfresco Case Study – American Association of Insurance Services
      • Alfresco Case Study – United Cerebral Palsy
    • HBase
    • DynamoDB
    • OpenText & Documentum Consulting
      • Upgrades – A Well Documented Approach
      • Life Science Solutions
        • Life Sciences Project Sampling
    • Veeva Consulting
    • Ephesoft
    • Workshare
  • Case Studies
    • White Papers
    • 11 Billion Document Migration
    • Learning Zone
    • Digital Asset Collection – Canadian Museum of Human Rights
    • Digital Archive and Retrieval – ASCP
    • Digital Archives – New York Philharmonic
    • Insurance Claim Processing – New York Property Insurance
    • Policy Forms Management with Machine Learning – AAIS
    • Liferay and Alfresco Portal – United Cerebral Palsy of Greater Chicago
  • About
    • Contact Us
  • Blog

Documentum to Portal Consistency Checker – Proof of Concept

You are here: Home / Documentum / Documentum to Portal Consistency Checker – Proof of Concept

February 29, 2012

TSG has several clients using Documentum as a repository and a custom front end application for consumption of the records or renditions of records.  In most cases there is a mechanism in place such as SCS (Site Caching Services) or TSG’s OpenMigrate PUMA (See CIS Case Study for more details).  While a typical Documentum application (ex: Webtop) provides a “one stop shop” for authors and approvers, the interface can be challenging when “consumers” are just looking for quick search and retrieval.  This solution provides improved performance, business continuity, and ability to add documents from other systems.  One potential risk to using a cache of documents and metadata for search and retrieval is the integrity of data.  Publishing techniques are designed to accurately cache records; however there are uncontrollable circumstances that may result in a mismatch.  

Some possibilities that could cause data inconsistencies between Documentum and the cache include:

  • Server outages
  • Database corruption
  • File system corruption
  • Poor or inaccurate database updates / administration
  • Documentum Errors
  • Neglected Development Environments

While these circumstances are unlikely the bottom line is there is no way to know that all the documents that should exist in the portal cache are all accurate, and all the documents that do not belong on the portal cache are removed.

It was this uncertainty that inspired the development for a proof of concept consistency checker program.  We wanted the program to be configurable, run on a schedule or as a Documentum job, evaluate some or all records (depending on cache size), optionally notify administrators of job results, and optionally fix any discrepancies on the spot that it may find.  In environments where there are simply too many records to process at once, we devised a system where the job will only process a portion of the documents on each run, but documents will not be re-inspected until all have been evaluated.

The code architecture is quite simple as the following pseudo code details:

  1. Query Source (Documentum) for all records or record subset – store to Map
  2. Query Target (Cache Database) for all records or record subset– store to Map
  3. Loop through Source record map and check for mismatches or missing Target records
  4. Loop through Target document map and check for missing Source records
  5. Send notification to administrator of all discrepancies (If enabled)
  6. Fix discrepancies (If enabled)

We tested the code on a development environment that we were aware had many issues.  The code processed 2,500 documents in 12 seconds and was able to identify and email a report of 67 inconsistencies.  We ran the code again with a parameter injected to actually fix the issues it located and that run completed in 45 seconds.

Filed Under: Documentum, OpenMigrate, Product Suite, R&D, TSG, Webtop Tagged With: Consistency, data integrity, Documentum, OpenMigrate, portal, PUMA, Webtop

Reader Interactions

Leave a Reply Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Primary Sidebar

Search

Related Posts

  • OpenMigrate: Bulk Load Interface Available for Download
  • Hadoop – Why Hadoop as a Content Store when Caching Content for ECM Consumers
  • Documentum or Alfresco – Redacting Sensitive Information with OpenRedact
  • Web Content Curating and Publishing (WCM) in Documentum and Alfresco
  • Documentum Migrations – OpenMigrate Sucessfully Moves Half Million Records
  • TSG Open Source Product Plans
  • Documentum Cross-Repository Searching – an integrated open source approach
  • Documentum – Replacing External SharePoint Sites with a Simple Cached Approach
  • Documentum 5 vs. 6, Databases and Dates: Does Anybody Really Know What Time It Is?
  • Documentum 6.5 Upgrade – Character Encoding Issues

Recent Posts

  • Alfresco Content Accelerator and Alfresco Enterprise Viewer – Improving User Collaboration Efficiency
  • Alfresco Content Accelerator – Document Notification Distribution Lists
  • Alfresco Webinar – Productivity Anywhere: How modern claim and policy document processing can help the new work-from-home normal succeed
  • Alfresco – Viewing Annotations on Versions
  • Alfresco Content Accelerator – Collaboration Enhancements
stacks-of-paper

11 BILLION DOCUMENT
BENCHMARK
OVERVIEW

Learn how TSG was able to leverage DynamoDB, S3, ElasticSearch & AWS to successfully migrate 11 Billion documents.

Download White Paper

Footer

Search

Contact

22 West Washington St
5th Floor
Chicago, IL 60602

inquiry@tsgrp.com

312.372.7777

Copyright © 2023 · Technology Services Group, Inc. · Log in

This website uses cookies to improve your experience. Please accept this site's cookies, but you can opt-out if you wish. Privacy Policy ACCEPT | Cookie settings
Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT