While working with a couple of clients recently the question of how best to organize documents in their ECM systems came up. Both had started organizing documents by department but then realized for the long-term that organizing by document type worked better. While no one-way of organizing information may fit every ECM implementation this post will share a few guidelines and best practices that make the long-term management of information a bit easier.
Generally, there are three commonly used document organizational models, or taxonomies, that are used by companies when setting up their ECM repository. (Please share your experience in the comments.)
- Organize documents by department
- Organize documents by business process
- Organize documents by document type
Document organization will influence the way applications access data and how records management policies are applied and maintained within the system. While records management retention policies can be applied regardless of document organization, the complexity increases the farther apart the file plan and document organization structures are categorically. We’ll cover more about records management in a future post.
In the case of applications, document organization particularly matters when data is presented for browsing to the user. Rarely does a user want to browse to an individual document. When looking for a particular document, searching is preferable. Browsing is better when looking for documents related to one another, such as all the documents related to an insurance claim. Even in this case, searching on an attribute (such as claim number) that is similar across documents to return a set of related information is more efficient than browsing.
If searching is more efficient why does it matter if documents are organized by department?
Departmental organization is the most volatile and is subject to frequent change, companies almost always start out with it first largely because most ECM implementations start out as departmental solutions.
ECM often starts within a single department at a company and those early adopters typically use their own roles/responsibilities and file shares to design the system. Quite often the documents are simply moved from the file share organization right into a repository. Initial implementations often clean up the data to remove the largest pain points found in a file share environment but it often remains grouped by who uses it.
This organization may work fine until someone leaves, a department is reorganized, or the need to share information outside the group arises. As soon as someone unfamiliar with the department needs to reference the documents the short comings appear. The new users may simply not be able to find what they need. This leads to frustration and a low adoption of the system by groups in other departments. In some cases, the new users will start forming their own departmental organization scheme and uploading duplicates of documents that already exist elsewhere in the system.
Is organizing by business process better?
Another common document organization is by business process. This is usually found when an ECM system has been implemented for a particular point solution and the documents are organized as needed for the business solution. Sometimes all the documents may be of the same type or owned by the same group, but not necessarily.
Occasionally, when documents are organized by business process they are not involved in the process as it is occurring but are instead stored in the ECM for archival search and retrieval. In this case, organizing for the process is not necessary since it is rarely navigated by the user. Rather the user searches for a particular document or set of documents. Financial documents, such as invoices, typically fall into this organization pattern.
In addition, unless the business process crosses departments, the documents may in fact be organized in a way that reflects the department executing the process. In essence, the repository is really using a departmental document organization model.
Why should documents be organized by type?
For ease of management and as a best practice, organizing by type facilitates maintenance of the repository and documents. A type-based organizational model can also be easily extended to accommodate new types and sub-types of documents.
The primary benefit is that type organization is durable. It survives company and department re-organizations and owners of types of documents can often easily be identified. With the need in most businesses to link document types to owners and record management file plans, organizing by type helps with managing information compliance.
Is document organization really relevant if my documents are attributed properly?
This is a very good question. In theory, no, the organizational structure should not matter if documents are attributed, or indexed, properly. However, in practice this is rarely the case.
Document organization provides context to the information. The browsing structure of the repository provides context. A set of search results also provides context. Depending on the events and actions that need to take place on the documents the decision between a browsing context versus a searching context may not be straightforward.
For example, as we have done more work with records management solutions, it has become apparent that the browsing (or foldering) path to a document is a significant component of the implementation affecting its performance.
In our HPI solution, we have actually mixed the two contexts to provide flexibility to the user. HPI Search provides a simple search on document and folder attributes; and HPI Desktop uses a root folder path as a starting point and then document attributes to dynamically create browse-able groupings of related documents.
Conclusion & Recommendations
Deciding how to organize documents often becomes an exercise in deciding between short-term need and long-term maintenance. Often, just getting information off a file server is the business driver and not much time is devoted to analyzing the information and deciding how to sort, filter, and import documents. Data cleaning is typically a pricy activity.
When possible, try organizing documents at a high level by type or owner. Perhaps start with only two levels, a document type/sub-type combination. Most users and owners can quickly group information in a simple sort. If needed for browsing or an application, a top-level folder based on the role of the document owner will help provide additional context. In many cases, if there is an urge to organize a document into a folder, the information may be better captured as an attribute on the document.
Please share your experience, how do you organize documents?
Great article on a topic that is so core to any ECM implementation yet there is hardly any best practices or case studies available on it
My experience has been similar to the recommendations in this article (I hope I have not misinterpreted anything)
1. Grouping by document type is perhaps the most suitable
2. Organising documents into folders does not matter – its actually recording the correct metadata because you could easily create the hierarchy dynamically – if the meta data is accurate (Google Documents follows a similar concept)
3. Organising by document type lends itself well to applying retention policies
4. Search is the most common way to access documents
=====
As an e.g. we implemented a Contracts Management solution for an organisation that was so decentralised and geographically dispersed and prone to org-restructures, mergers and acquisitions
The Legal department was a shared service across the business – globally
Multiple businesses within the company could jointly provide a service to a customer and in other instances each business could have their own private contracts
There were confidential agreements e.g. for directors which only certain people could see and then there were enterprise wide agreements that were struck by the corporate department and not available to the individual businesses
—
In this situation – organising contracts by Org Structure (akin to the department model) with the traditional ‘folder inheritance’ security model just would not fly
In-fact we found that organising the contracts chronologically (with security policies applied dynamically to each agreement) was the best approach for those rare instances where the user would browse to find an agreement.
Search was the most common way to access agreements
The system had a rich meta-data model – which did present some user adoption issues – but in the end the solution could be used globally despite all the diversity
Hi Paras
Thank you for the reply. You are spot on and provided a great example for why it is critical to know both your documents and your users when designing the repository.
This is a really useful blog posting.
Your reason does make sense. But if we don’t structurize as organization department, that would be very difficult when setting the permission access.
If it is based on content type, everytime we upload we need to defnine the permission access which groups/users that has access. It would be different case if we organize it based on department.
Please share your opinion about this
Hi Tiur –
You are correct that security in many companies is tied to how the documents are organized. Setting up security by department is typically the path of least resistance. It is easy for people to understand and usually reflects the security that was on a file share.
One of the key challenges when designing a security model is to keep it as simple as possible and no simpler. It is very easy to over-think security. When developing a model try to think of security from a cross-departmental or process perspective. I find that the 80/20 rule often applies. Basically, 80% of the documents can be accessed by most everyone in a company. The remaining 20% are typically of a specific type, such as secured invoices or HR information. Those can usually be identified and secured by type. If its just not possible to define a security model by type, then assigning securiity using a rule or SBO/TBO is an option.
Christine