One of the more common requirements we see from our manufacturing and operation clients is tight control around document versioning. In some cases, for example Pharmaceuticals, these systems are tightly regulated (and possibly audited) by a government agency such as the FDA. As we see more and more clients choosing Alfresco over Documentum, this post will share our thoughts on compliance requirements as well as how each product handles requirements around versioning and security.
Versioning, Security, and Relationships – Differences between Alfresco and Documentum
To illustrate the differences, consider a common document scenario:
- A Standard Operating Procedure (SOP) document exists in the system called ACME-1234. This document describes the bottling process for a product manufactured at a plant. The document is version 1.0 and Effective in the system.
- Pam, a Quality Assurance lead, learns that 1 out of every 10,000 bottles have a leaky cap. Jim and his team identify a fix that will reduce the chances of caps leaking. As part of this process, the ACME-1234 SOP must be updated.
- Jim, a Document Editor on Pam’s team, edits ACME-1234 and checks in a new copy, version 1.1 in the Draft state.
- Mike, a plant supervisor, executes a search for SOPs in the system related to his area of the plant. When ACME-1234 is returned in the search results, only version 1.0 is listed. The 1.1 Draft version must NOT appear in the search results since it is not yet approved.
- When Jim or Pam run a search for ACME-1234, both version 1.0 and 1.1 are visible.
- Jim and Pam work together to edit ACME-1234 and eventually have a version 1.3 Draft. Satisfied with the changes, Pam routes the document for approval to the appropriate document approvers.
- During the review and approval process, Dwight adds Annotations to version 1.3 and rejects the updates. Pam receives a notification, reviews Dwight’s comments, makes the necessary updates, and versions the document to 1.4. Pam re-starts the approval process on version 1.4. The annotations added to 1.3 are still visible, though they are tied to the 1.3 version and not carried forward to 1.4.
- Throughout this process, when Mike runs his searches, he continues to see ACME-1234 version 1.0 in the results. Mike does not see the in process versions.
- Once all document approvers approve the change, version 2.0, Approved is created and has an effective date of two weeks from the approval date.
- When Mike runs his search, he now sees both version 1.0, Effective and 2.0, Approved. Mike and his team have two weeks to train on the new SOP.
- After two weeks are up, ACME-1234 version 2.0 is promoted to Effective. Version 1.0 is Superseded.
- When Mike runs his search, he now only sees version 2.0, Effective. When Pam runs the same search, she sees both version 2.0, Effective and 1.0 Superseded.
Given the above scenario, let’s look at how we can satisfy the requirements relating to versions, security, and relationships with both Documentum and Alfresco.
Versioning and Security
In Documentum, each version of an object is a first class citizen. It has it’s own ID and the ability to have it’s own independent ACL security set. So in the above scenario, SOP-1234 version 1.0, Effective can have the Effective Document ACL, and version 1.1, Draft can have the Draft Document ACL. Depending on the security configurations, only certain users will see Draft and in process documents. The above scenario can be handled out of the box with Documentum versioning and security.
In Alfresco, when a document is versioned, a new ID is not assigned to the new version. Instead, the new version has the same ID and the old version exists as a “snapshot” in the version store. It is impossible in Alfresco to have different security sets applied across two versions of the same document. Additionally, it is impossible to run a search that returns both the latest version of a document as well as previous versions. To accommodate the scenario above, extensive behaviors must be coded in Java. Essentially, the code must create a new repository object when version 1.0 is versioned to 1.1. This allows the Draft version 1.1 to have a different security set than the Effective 1.0. A “supersedes” association should also be created to track the relationship between the objects. The user interface must be aware of this relationship so the document can be presented as one continuous list of versions to the end user rather than two separate document objects.
TSG’s HPI, Active Wizard and OpenContent products contain the necessary behaviors to enable these requirements for Alfresco.
Versioning and Associations
Let’s take a deeper look at the steps in the scenario above where a reviewer adds annotations to a document and then the document is versioned. In this scenario, the annotations should only be contained on the version the user annotated. When the document is versioned, the annotations should “stick” to the prior version.
In Documentum, a relationship is tied to from document version to document version. Therefore, we can easily tie the annotations to version 1.3 of ACME-1234. when version 1.4 is created, the annotations “stick” to version 1.3 and are not carried forward to version 1.4.
In Alfresco, relationships are from document to document, not version to version. This means that when annotations are added to 1.3, if no custom code is added, the annotations will still exist on the 1.4 version after the document is versioned.
The above difference in how associations are handled with Documentum vs. Alfresco has other implications as well. For example, typically a Change Request form is associated to the document as it goes through the change process. The requirements typically state that the form is attached to the minor version, in our case it is 1.4 and the association sticks onto the latest minor version until the document reaches 2.0. Once the document reaches 2.0, the form association is “frozen”, meaning the 2.1 version should not have a Change Request association. This scenario is not possible with Alfresco out of the box.
As with the versioning and security discussion above, TSG’s HPI, Active Wizard, OpenAnnotate, and OpenContent work together to give Alfresco the functionality to satisfy these requirements. For additional information, see our compliance solution for Documentum, Alfresco or Hadoop.
Let us know your thoughts below:
>Effective can have the Effective Document ACL, and version 1.1, Draft can have the Draft Document ACL. Depending on the security configurations, only certain users will see Draft and in process documents. The above scenario can be handled out of the box with Documentum versioning and security
It is a worst practice ever.
Andrey,
Not sure what you mean by this being a bad practice. I’m assuming you don’t mean the business requirement of some users seeing the 1.0 Effective and not being able to access the 1.1 Draft. Perhaps the post doesn’t go in-depth enough with how we are meeting this business requirement with out of the box Documentum lifecycles and ACLs.
In the scenario you quoted, we have two ACLs. Let’s call them quality_doc_effective and quality_doc_draft. The appropriate groups have been assigned read, write, none, etc. permissions. We also have a lifecycle policy that’s applied to all controlled documents, let’s call it controlled_doc_policy.
In the controlled_doc_policy lifecycle configuration, we have a number of states, but let’s concentrate on Draft and Effective. In the Draft state, we configure that the document should have the quality_doc_draft ACL applied. In the Effective state, the document should have the quality_doc_effective ACL. We also configure that if an Effective document is checked out and checked in, the checked in version should be demoted to the base state, which is Draft.
So here’s what happens. A document is sitting at 1.0, Effective and has the quality_doc_effective ACL. The appropriate people can see the document. More importantly, the appropriate people can check out the document to make a change. This is important because I’m sure you would be upset if any user could just go in to Documentum and make a random update to a Nuclear Procedure or drug manufacturing SOP.
Someone makes a change to the document and checks in version 1.1. The lifecycle configurations state that version 1.1 will now have the quality_doc_draft ACL. So basically, we’ve met a complex business requirement with out of the box Documentum features and a configuration-only approach.
Perhaps you were thinking that we were *generating* a new ACL for every version? If that’s the case, a) we don’t do that, and b) I would agree with you – creating a new ACL for every version is definitely a bad practice.
Does the above information clear up the issue for you? If not, what approach do you suggest?
The problem is your case is extremely limited – you are considering the case when one document obsoletes another, moreover, you want to stay within “out of the box Documentum features”, actually, it is not clear what did you mean under this term – I believe, you meant webtop, because for other products I do not see any problem to add pair of attributes obsoletes/obsoleted_by to custom object type and manage “your versions” as different documents. Why is your case limited? What are you going to do if one document obsoletes a set of documents or vice versa – set of documents obsoletes one document? Version approach won’t work in such case and anyway you will need to maintain obsoletes/obsoleted_by relationship. Just consider this as a quick counterexample to your approach, but the general idea is: metadata versioning cause a lot of pain i behind, try to avoid it.
I’m not talking about document obsoletion, just versioning a document. Simple checkout / checkin. The business case is that there needs to be different permission on the two versions. I’m not trying to describe the entire business process here, just this one part. The point of this post is to explore how Alfresco and Documentum differ when it comes to versioning, security, and relationships – not an explanation of the complete solution.
I’m not sure what you mean by metadata versioning – we’re certainly not doing that. In compliance parlance, we’re superseding versions. Eventually version 2.0 supersedes 1.0. The data as to why the change happens, what other documents are affected, etc. would be stored elsewhere. Some clients store that information in Change Request forms, other clients automate reports, etc.
Obsoletion in the compliance scenario is a totally different beast. We have a separate process for document obsoletion since in most industries deleting one of these documents is not allowed. You can see it in action in Demo #8 here: https://tsgrp.wpengine.com/solutions/compliance-for-regulated-industries/demo-series/
> The business case is that there needs to be different permission on the two versions.
That is definitely not a business case, business case sounds like “regular users do not have access to drafts and pending documents”, what you mean under business case/requirements is in fact a functional design specification, i.e. you already processed business requirements and decided how are you going to implement these requirements, but what you are trying to present as advantage of Documentum (i.e. different metadata for each version and ability to create relationships between versions) in real life produces a lot of mess in repository.
> Eventually version 2.0 supersedes 1.0
On your videos I do see that reviewers/approvers review/approve version 1.1, at the end of reviewing process version 1.1 somehow turns into version 2.0 – you already had written custom code, there is no point to insist that it is a good idea to stay within “out of the box Documentum features”: want to run approval process for new version of document? Ok, create a copy of file, approve and upon finish attach it as new version to already exisiting document.
I’m trying to stay out of the box when it comes to document versioning and security, but yes, there are many components that go into the overall solution (TBOs, lifecycle SBOs, etc). For example, you mention the document somehow getting to 2.0 – a lifecycle SBO attached to the Approved state handles that.
I think we’re going to have to agree to disagree here. Documentum provides robust support for document versioning, metadata on versions, lifecycle and security. I’d prefer to use those tools rather than work around them.
> there are many components that go into the overall solution (TBOs, lifecycle SBOs, etc)
> Documentum provides robust support for document versioning, metadata on versions, lifecycle and security
I believe you favourite Documentum errors are: DM_POLICY_E_EXIST_NO_BP_LOG, DM_POLICY_E_METHOD_TIMEOUT and DM_POLICY_E_ERROR_NNNN 🙂 Long time ago we used to use Documentum lifecycles, but now I consider such approach as a bad practice as well.
We have many clients using Documentum lifecycles with success. Of course, the JMS needs to be healthy or those errors would crop up.
What? Then how would you support updating, reviewing, approving, and issuing controlled documents in the oil & gas or resources industries?
Simon – Not sure if you’re replying to my comment or Andrey’s. If mine, we would support those industries in a similar manner as the manufacturing and nuclear use cases. Perhaps these links make the solution more clear:
https://tsgrp.wpengine.com/solutions/compliance-for-regulated-industries/
https://tsgrp.wpengine.com/solutions/compliance-for-regulated-industries/demo-series/
Are there special requirements in the oil and gas industry that differ from the business scenario in the post? If so, can you explain what the differences are?
To Andrey’s. I totally get where you were coming from.