TSG recently worked with a client evaluating EMC’s Retention Policy Services (RPS) product. For those unfamiliar with RPS, RPS is used for automating content retention and disposition, while also allowing for holds that prevent disposition. Additional information can be found on the EMC website here: http://www.emc.com/products/detail/software2/retention-policy-services.htm
The client liked many of the features of the product and its user interface (which is very similar to Webtop). One of the appealing features is that RPS allows a clear separation of duties between roles of users along with multiple tools for managing retention with the WDK-based Retention Policy Services Administrator (RPSA). In this case, the end users are never aware that RPS is being used unless they try to delete a document that has a retention policy applied. For this client, a major concern focused on the RPS performance overhead of applying retention policies and mark ups to a multi-million document repository.
RPS was designed to be applied to many different types of document retention situations. To give RPS its flexibility, the product creates and maintains new RPS objects and audit trail events for each object under retention. Retention policies are applied to objects, and the policy denotes the length of time until the next action is invoked (culminating in disposition).
Typically a retention policy is applied to a folder where it can then be inherited without instantiating retainer objects to the documents in the folder. With this approach, if a folder had 100 objects or 3 objects there would only be one RPS retainer object. If a retention policy is applied at the document, or individual, level then there is a 1:1 ratio with the RPS retainer object. For the client, this implied that if we had one million document objects we would also have one million RPS retainer objects. This drastically increases the number of retainer objects and the overhead in the repository.
An Alternative Solution?
With a better understanding of the RPS performance impact to the repository, our client is determining how best to handle other RPS features that might need to be customized due to the large number of documents, as well as the restrictions placed on end users regarding deletion of their documents ahead of RPS schedule.
- For approval of disposition, when a document is ready to be approved, the user is sent a notification. Our client was worried about getting hundreds of notifications (and emails) on any given day. We were thinking of a way to do this more in a bulk mode where a user could approve large batches of documents all at once rather than hundreds of emails and inbox entries via a Webtop customization (the RPS Engineer was intrigued by this idea!)
- For postponing disposition, the client was looking at a way to not require end users to make use of markups (which normally can only be done by an administrative role within RPSA), but instead use Webtop to postpone disposition until a later date when they are prompted again.
- For deletion, documents in a specific folder are earmarked for disposition after a certain length of time. With this approach, If a user wants to delete a document sooner it is not an issue since the retention policy is geared more toward making sure unnecessary documents are deleted. Using an alternative approach resolves the restriction in RPS where only a user in the Retention Manager role has access to delete the document.
Implementing Alternatives for Document Retention
In working with clients evaluating RPS, some have decided to go with a simple custom solution instead of RPS for a combination of reasons including functionalty as well as licensing. Typically, this type of solution revolves around adding an additional attribute to the existing object type to denote the retention date, and running a job at set intervals that determined which objects need to be disposed of based on the attribute (and generated a list of these objects ahead of time for anyone to review).
Paras Jethwani says
If possible – can you explain the ‘solution around deleting documents under retention – sooner than the disposition date’ in a bit more detail please.
In this particular case, the client doesn’t mind users deleting documents ahead of the policy disposition date since the retention policy is the max amount of time a document should be kept. To implement this, a check in Webtop is performed before the destroy is called to check security for the user on the document and if a retainer is applied. If the preconditions are met, and there is no retention markup applied to the document, a service account with Retention Manager role would be used to delete the retainer and subsequently delete the document.
Thanks for replying!
chris mccabe says
RPS and Records Manager can really suck to work with in all but the most vanilla straightforward situations and where internal corporate records roles align fairly nicely (fat chance) with the built-in RPS/RM roles. At least until 6.5 sp2, (I can’t speak for versions after that), they were very difficult products to customize with and were barely supported at the API level. Forget helpful documentation.
Your alternative solution is a nice one in practice but too often companies want a fast solution and are sold on RPS and/or RM as part of a package deal or are willed into implementing it for fear they may not be in compliance and need the stamp of an authoritative source (EMC) to sign off that they have all the right capabilities in the Recors world.
Putting the retainer at the folder level or customizing to apply a retainer automatically once the document is saved is typical. My experience is, most records dept personnel have sbsolutely no idea what they’re getting themselves into with the RPS/RM stuff and quickly find their internal jobs/roles co-opted into some kind of quasi RM/RPS “administrator” in addition to their regular jobs. Having trained and professional DCTM admins and developers on site who are dedicated to help out will save a lot of pain. I guess it’s the lesser of two evils – the pain of implementing RM/RPS (or for that matter, Federated Retention Services) versus getting into hot water cuz some bozo deleted a court ordered document.
Paras Jethwani says
Hi Chris M,
“Putting the retainer at the folder level or customizing to apply a retainer automatically once the document is saved is typical” – Just wondering if retention cascading take care of this without needing any customisation?
e.g. you could save a document in a folder hiearchy that is more suitable/accessible to end users and also link it to a folder/file-plan (in the background) which has the appropropriate retention policy applied.
** I am also thinking in the context of an xCP workflow – where a completed case-folder is linked to a file-plan/folder as the last step in the workflow **
Would be interested to hear your thoughts on this
Your example of linking a document to two folders is precisely a scenario we examined. The design and management of a shadow folder structure as well as the implementation of the rules and triggers to place the documents was very complex.
I would be interested in hearing more about anything you have done with this approach.
Mike Clarke says
RPS RM and especially PRM require specialized skills, in addition to Documentum so you should expect to have a records specialist on staff in addition to Documentum to manage the retention policies, etc.
‘Putting the document under the required approvel in the final phase of the policy is not generating the notification to the approver’.
wondering to hear your thoughts on this….
I have an issue that the retention is not getting inherited to scanned documents. It creates folders and documents in Documentum through an external application, and retention gets applied to the top level folder but won’t get inherited to the documents. Any thoughts? btw we using only 6.5 sp3 RPS with no RM
Without knowing more about the system I can only speculate. However, I suggest you post this question on the EMC Documentum support forums or open a support request with EMC as well. In the meantime a couple of things to check would be to see if you have folder or document level retention policies set up. Also, has this ever worked before and if so, what might be different now? Another thing you can try is to turn on DFC tracing and see if you are getting any error messages that perhaps are not bubbling all the way up through your application.
Can you checkout a document under retention and check it back in a new revision?
Yes you can check out a document under retention and save it back as a new version.
Akansh Katyayan says
From a performance stand point what is the best way to apply retention to documents. We have to apply retention to about 20 million documents at a time. Currently we are using Java Code to apply retention on documents using sysObj.link, But performance is not what we expect. Please suggest alternative way