Capture 2.0 – Metadata Extraction with Machine Learning Upon Ingestion

TSG is predicting future disruptions to content capture within the ECM industry. In the 4th quarter of 2019, we focused on improving the OpenContent Management Suite  by disrupting legacy capture solutions with machine learning. As we are predicting that even more customers will be moving to all digital documents given the pandemic, our Capture 2.0 efforts for 1st quarter 2020 have focused on integrating OpenMigrate into the ecosystem in order to extract metadata as the document is ingested into the system. Utilizing this approach, the user would then verify the index metadata in OCMS before finalizing the document in the repository.

Capture 2.0 Machine Learning

Previous Capture 2.0 posts have referred to the following diagram:

1. Create and Train – Capture administrators will be able to create initial templates with extraction rules (ex: zonal, key/value pair, etc). These templates will be fed into the suggestion engine
2. Bulk Ingestion – As documents enter the system, OpenMigrate can call the suggestion engine to classify documents and extract metadata.
3. Store Completed Docs – After receiving the extracted data, if required fields are all filled with a high enough confidence level, the document is filed in the repository in the correct location.
4. Queue Incomplete Docs – If all required fields cannot be completed with high enough confidence, the document is placed into the repository and queued for indexing in OCMS.
   • Note that in either case above, the document is always ingested to the repository.
5. Extract Metadata – During OCMS indexing, the suggestion engine can be called to return metadata suggestions for documents that have not yet been processed through the suggestion engine. This can happen, for example, for documents that were queued for indexing by a process other than OpenMigrate.
6. Finalize Document – the user works through the queue of documents to index, verifying the metadata suggestions extracted from the document and saving the final metadata values.
7. Extraction Error Corrections – during the previous step, the indexing module of OCMS keeps track of any error corrections that were made. For example, if the user dismisses one of the original suggestions and selects a different value on the document, that correction is fed back into the suggestion engine so that the next time a similar document is processed, the same mistake is not repeated.

In our prior post, we’ve focused on step 1 (creating and training) as well as steps 5-7 (indexing and feedback loop). In this post, we are focusing on steps 2-4 by integrating OpenMigrate into the Capture 2.0 ecosystem.

Scenario Overview

Many customers accept invoices and other documents from vendors and 3rd parties via email or other electronic ingestion methods. Since OpenMigrate can easily monitor an email inbox and ingest attachments, it can reach out to the Capture 2.0 suggestion engine as part of the OpenMigrate process. After receiving metadata values from the suggestion engine, OpenMigrate then queues the document for review within OCMS.

The video below overviews this process and shows how the system can allow the user to simply verify the indexing information for the already learned invoices as well as teach the system where indexing information exists on new vendor invoices that the system has not yet learned.

Let us know your thoughts below: