10%

Try : Insurtech, Application Development

Edtech(5)

Events(31)

Interviews(10)

Life@mantra(10)

Logistics(1)

Strategy(14)

Testing(7)

Android(43)

Backend(28)

Dev Ops(2)

Enterprise Solution(20)

Frontend(28)

iOS(39)

Javascript(13)

AI in Insurance(27)

Insurtech(57)

Product Innovation(36)

Solutions(15)

Augmented Reality(7)

Customer Journey(7)

Design(6)

User Experience(21)

Artificial Intelligence(95)

Bitcoin(7)

Blockchain(14)

Cognitive Computing(7)

Computer Vision(6)

Data Science(14)

FinTech(41)

Intelligent Automation(25)

Machine Learning(43)

Natural Language Processing(10)

E-health(3)

HealthTech(5)

mHealth(3)

Telehealth Care(1)

Tabular Data Extraction from Invoice Documents

5 minutes, 12 seconds read

The task of extracting information from tables is a long-running problem statement in the world of machine learning and image processing. Although the latest accomplishments in the field of deep learning have seen a lot of success, tabular data extraction still remains a challenge due to the vast amount of ways in which tables are represented both visually and structurally. Below are some of the examples: 

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Invoice Documents

Many companies process their bills in the form of invoices which contain tables that hold information about the items along with their prices and quantities. This information is generally required to be stored in databases while these invoices get processed.

Traditionally, this information is required to be hand filled into a database software however, this approach has some drawbacks:

1. The whole process is time consuming.

2. Certain errors might get induced during the data entry process.

3. Extra cost of manual data entry.

 An invoice automation system can be deployed to address these shortcomings. The idea is to upload the invoice document and the system will read and generate the tabular information in the digital format making the whole process faster and more cost-effective for companies.

Fig. 6

Fig. 6 shows a sample invoice that contains some regular invoice details such as Invoice No, Invoice Date, Company details, and two tables holding transaction information. Now, our goal is to extract the information present in the two tables.

Tabular Information

The problem of extracting tables from invoices can be condensed into 2 main subtasks.

1. Table Detection

2. Tabular Structure Extraction.

 What is Table Detection?

 Table Detection is the process of identifying and locating tables that are present in a document, usually an image. There are multiple ways to detect tables in an image. Some of the approaches make use of image processing toolkits like OpenCV while some of the other approaches use statistical models on features extracted from the documents such as Text Position and Text Characteristics. Recently more deep learning approaches have been used to detect tables using trained neural networks similar to the ones used in Object Detection.

What is Table Structure Extraction?

Table Structure Extraction is the process of extracting the tabular information once the boundaries of the table are detected through Table Detection. The information within the rows and columns is then extracted and transferred to the desired format, usually CSV or Excel file.

Table Detection using Faster RCNN

Faster RCNN is a neural network model that comes from the RCNN family. It is the successor of Fast RCNN created by Ross Girshick in 2015. The name Faster RCNN is to signify an improvement over the previous model both in terms of training speed and detection speed. 

To read more about the model framework, one can access the paper Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.

 There are many other object detection model architectures that are available for use today. Each model comes with certain advantages and disadvantages in terms of prediction accuracy, model parameter size, inference speed, etc.

For the task of detecting tables in invoice documents, we will select the Faster RCNN model with FPN(Feature Pyramid Network) as a feature extraction network. The model is pre-trained on the ImageNet corpus using ResNET 101 architecture. The ImageNet corpus is a public dataset that consists of more than 20,000 image categories of everyday objects.  We will therefore make use of a Pytorch framework to train and test the model.

The above mentioned model gives us a fast inference time and a high Mean Average Precision. It is preferred for cases where a quick real time detection is desired.

First, the model is to be trained using public datasets for Table Detection such as Marmot and UNLV datasets. Next, we further fine-tune the model with our custom labeled dataset. For the purpose of labeling, we will follow the COCO annotation format.

Once trained, the model displayed an accuracy close to 86% on our custom dataset. There are certain scenarios where the model fails to locate the tables such as cases containing watermarks and/or overlapping texts. Tables without borders are also missed in a few instances. However, the model has shown its ability to learn from examples and detect tables in multiple different invoice documents. 

Fig. 7

After running inference on the sample invoice from Fig 6, we can see two table boundaries being detected by the model in Fig 7. The first table gets detected with 100% accuracy and the second table is detected with 99% accuracy.

Table Structure Extraction

Once the boundaries of the table are detected by the model, an OCR (Optical Character Reader) mechanism is used to extract the text within the boundaries. The text is then processed using the information that is part of a unique table.

We were able to extract the correct structure of the table, including its headers and line items using logics derived from the invoices. The difficulty of this process depends on the type of invoice format at hand.

There are multiple challenges that one may encounter while building an algorithm to extract structure. Some of them are:

  1. The span of some table columns may overlap making it difficult to determine the boundaries between columns.
  2. The fonts and sizes present within tables may vary from one table to another. The algorithm should be able to accomodate for this variation.
  3. The tables might get split into two pages and detecting the continuation of a table might be challenging.

Certain deep learning approaches have also been published recently to determine the structure of a table. However, training them on custom datasets still remains a challenge. 

Fig 8

The final result is then stored in a CSV file and can be edited or stored according to one’s convenience as shown in Fig 8 which displays the first table information.

Conclusion

The deep learning approach to extracting information from structured documents is a step in the right direction. With high accuracy and low running time, the systems can only learn to perform better with more data. The recent and upcoming advancements in computer vision approaches have made processes such as invoice automation significantly accessible and robust.

About the author:

Prateek Sethi is a Data Scientist working at Mantra Labs. His work involves leveraging Artificial Intelligence to create data-driven solutions. Apart from his work he takes a keen interest in football and exploring the outdoors.

Further Reading:

Cancel

Knowledge thats worth delivered in your inbox

How to increase patient engagement on a healthcare app?

By :
4 minutes, 9 seconds read

Patient facing mobile apps have emerged as a viable alternative to interact with patients and help them execute several tasks related to healthcare without reaching the hospital. Patients are now increasingly glued to these healthcare apps and their time spent on mobile devices has increased significantly. From communication and information to executing business through mobile apps have become a common thing.

Still, there is a lot of gap in the healthcare market. Marketers are struggling to gauge the true value of mobile apps for the bottom line of healthcare organizations. Especially in terms of the value of the app viz-viz the efforts, cost, and time it would take to become a useful and engaging healthcare app for the patient.

To help you increase the patient engagement on a healthcare app, here are a few crucial ways that you need to know. It will not only increase the engagement but will also help you meet your organizational goals at a faster pace.

1. Reduce Readmissions

Healthcare provider’s biggest pain point these days is to avoid readmission of preventable cases. It is crucial for healthcare organizations to reduce this number to enhance patient outcome and improve their revenue growth. Customized healthcare applications can assist hospitals in reducing the number of readmissions.

Here is how:

  • Offer personalized post-release information and instructions
  • Reminder for regular follow-up visits
  • Enforce stricter adherence to post-release prescription and regiment
  • Easier access to healthcare resources and information
  • Lesser cost and time of reaching out
  • Greater engagement to reduce readmission rates

2. Encourage Patients to Proactively Manage Their Health

Encouraging the patient to become proactive in managing their own health can help in improving the outcome and also enhance the reputation of your hospital. With easier access to the required resources and tools, patients are more likely to stay in touch with healthcare professionals and practice the wellness regimen. A properly optimized mobile app can deliver a better wellness experience to the patient and a greater sense of satisfaction.

Here is how an app can help the patient in managing their health proactively:

  • Keeps the patient informed and connected with relevant services
  • Encourages regular health tracking
  • Helps in developing healthy habits and exercises
  • Promotion of health education by streaming informations

3. Improve Trust and Build Relationships


Establishing trust between the patient and doctor is one of the difficult things that hospitals face. Due to long wait times, complex processes, or lack of communication between healthcare team; patients are not willing to attend healthcare appointments.

However, when you give all the relevant tools and information in the smartphone of the patient and empower them, the trust develops between the two parties and the patient proactively takes charge of their treatment. Hospitals can also gain competitive advantage by streamlining patient referrals and building stronger relationships with physicians.

Here is how apps can help:

  • You can provide access to a larger pool of specialists 
  • Help in easily accessing credentials, studies, and information from the mobile app
  • Recommend, network, and connect
  • Improve efficiency and workflow

4. Boost Brand Image and Reputation

Today patients can not be treated less than consumers. Hospitals are competing with each other to provide better care facilities and infrastructure at affordable prices. Hence, it has become crucial to achieve patient satisfaction and engagement. Through mobile apps, hospitals can make the existing information easily accessible along with brand awareness features including social media, photo galleries, virtual tours, and more.

Here is how mobile app can help:

  • Easier access to communication and information
  • Intuitive presentation of the hospital through immersive galleries
  • Stream ER wait time and other relevant information
  • Greater social media engagement

What more factors can increase engagement?

Several patients come to healthcare app once and then dump them after a few logins. One of the major reasons for such low engagement rates is that either the app is difficult to navigate or it is not immersive for the patient.

Consider these few points to make your app more engaging:

  • Customize the app as per the patient group. An app can appear differently to a child and an old age person. Develop an app that is easy to use for all users. Sit with your QA engineers to validate the functionality of your application thoroughly.
  • Build an app that is scalable and can be evolved over a period of time. Leave the possibility of enhancements and customizations that you would need in the future to keep the application viable.
  • It is a no brainer to give multiple options to your patients. Develop a cross-platform application that is compatible with both iOS and Android platforms and offers a rich user experience.

Wrapping Up

With the COVID-19 pandemic, people are more worried about visiting hospitals. Social distancing has become the norm and patients are more inclined towards telemedicine for their treatment. In such a tricky situation, it has become more crucial for hospitals to provide a robust, secure, and engaging mobile app to patients to interact with doctors, access information, and stay connected with the healthcare system.

About the Author

Erna Clayton is a techie with over 12 years of experience in several technological domains including quality assurance and software testing. In her free time, she loves travelling and writing on technology.

Further Readings:

  1. Building Consumer Trust in the Digital Healthcare Era
  2. HealthTech 101: How are Healthcare Technologies Reinventing Patient Care
  3. Virtual health: Delivering care through technology
  4. How Mobile Micro-Health Insurance can unlock ‘Digital for Bharat’?
Cancel

Knowledge thats worth delivered in your inbox

Loading More Posts ...
Go Top
bot

May i help you?

bot shadow