Try : Insurtech, Application Development

AgriTech(1)

Augmented Reality(21)

Clean Tech(9)

Customer Journey(17)

Design(45)

Solar Industry(8)

User Experience(68)

Edtech(10)

Events(34)

HR Tech(3)

Interviews(10)

Life@mantra(11)

Logistics(6)

Manufacturing(3)

Strategy(18)

Testing(9)

Android(48)

Backend(32)

Dev Ops(11)

Enterprise Solution(33)

Technology Modernization(9)

Frontend(29)

iOS(43)

Javascript(15)

AI in Insurance(39)

Insurtech(67)

Product Innovation(59)

Solutions(22)

E-health(12)

HealthTech(24)

mHealth(5)

Telehealth Care(4)

Telemedicine(5)

Artificial Intelligence(153)

Bitcoin(8)

Blockchain(19)

Cognitive Computing(8)

Computer Vision(8)

Data Science(23)

FinTech(51)

Banking(7)

Intelligent Automation(27)

Machine Learning(48)

Natural Language Processing(14)

expand Menu Filters

Tabular Data Extraction from Invoice Documents

5 minutes, 12 seconds read

The task of extracting information from tables is a long-running problem statement in the world of machine learning and image processing. Although the latest accomplishments in the field of deep learning have seen a lot of success, tabular data extraction still remains a challenge due to the vast amount of ways in which tables are represented both visually and structurally. Below are some of the examples: 

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Invoice Documents

Many companies process their bills in the form of invoices which contain tables that hold information about the items along with their prices and quantities. This information is generally required to be stored in databases while these invoices get processed.

Traditionally, this information is required to be hand filled into a database software however, this approach has some drawbacks:

1. The whole process is time consuming.

2. Certain errors might get induced during the data entry process.

3. Extra cost of manual data entry.

 An invoice automation system can be deployed to address these shortcomings. The idea is to upload the invoice document and the system will read and generate the tabular information in the digital format making the whole process faster and more cost-effective for companies.

Fig. 6

Fig. 6 shows a sample invoice that contains some regular invoice details such as Invoice No, Invoice Date, Company details, and two tables holding transaction information. Now, our goal is to extract the information present in the two tables.

Tabular Information

The problem of extracting tables from invoices can be condensed into 2 main subtasks.

1. Table Detection

2. Tabular Structure Extraction.

 What is Table Detection?

 Table Detection is the process of identifying and locating tables that are present in a document, usually an image. There are multiple ways to detect tables in an image. Some of the approaches make use of image processing toolkits like OpenCV while some of the other approaches use statistical models on features extracted from the documents such as Text Position and Text Characteristics. Recently more deep learning approaches have been used to detect tables using trained neural networks similar to the ones used in Object Detection.

What is Table Structure Extraction?

Table Structure Extraction is the process of extracting the tabular information once the boundaries of the table are detected through Table Detection. The information within the rows and columns is then extracted and transferred to the desired format, usually CSV or Excel file.

Table Detection using Faster RCNN

Faster RCNN is a neural network model that comes from the RCNN family. It is the successor of Fast RCNN created by Ross Girshick in 2015. The name Faster RCNN is to signify an improvement over the previous model both in terms of training speed and detection speed. 

To read more about the model framework, one can access the paper Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.

 There are many other object detection model architectures that are available for use today. Each model comes with certain advantages and disadvantages in terms of prediction accuracy, model parameter size, inference speed, etc.

For the task of detecting tables in invoice documents, we will select the Faster RCNN model with FPN(Feature Pyramid Network) as a feature extraction network. The model is pre-trained on the ImageNet corpus using ResNET 101 architecture. The ImageNet corpus is a public dataset that consists of more than 20,000 image categories of everyday objects.  We will therefore make use of a Pytorch framework to train and test the model.

The above mentioned model gives us a fast inference time and a high Mean Average Precision. It is preferred for cases where a quick real time detection is desired.

First, the model is to be trained using public datasets for Table Detection such as Marmot and UNLV datasets. Next, we further fine-tune the model with our custom labeled dataset. For the purpose of labeling, we will follow the COCO annotation format.

Once trained, the model displayed an accuracy close to 86% on our custom dataset. There are certain scenarios where the model fails to locate the tables such as cases containing watermarks and/or overlapping texts. Tables without borders are also missed in a few instances. However, the model has shown its ability to learn from examples and detect tables in multiple different invoice documents. 

Fig. 7

After running inference on the sample invoice from Fig 6, we can see two table boundaries being detected by the model in Fig 7. The first table gets detected with 100% accuracy and the second table is detected with 99% accuracy.

Table Structure Extraction

Once the boundaries of the table are detected by the model, an OCR (Optical Character Reader) mechanism is used to extract the text within the boundaries. The text is then processed using the information that is part of a unique table.

We were able to extract the correct structure of the table, including its headers and line items using logics derived from the invoices. The difficulty of this process depends on the type of invoice format at hand.

There are multiple challenges that one may encounter while building an algorithm to extract structure. Some of them are:

  1. The span of some table columns may overlap making it difficult to determine the boundaries between columns.
  2. The fonts and sizes present within tables may vary from one table to another. The algorithm should be able to accomodate for this variation.
  3. The tables might get split into two pages and detecting the continuation of a table might be challenging.

Certain deep learning approaches have also been published recently to determine the structure of a table. However, training them on custom datasets still remains a challenge. 

Fig 8

The final result is then stored in a CSV file and can be edited or stored according to one’s convenience as shown in Fig 8 which displays the first table information.

Conclusion

The deep learning approach to extracting information from structured documents is a step in the right direction. With high accuracy and low running time, the systems can only learn to perform better with more data. The recent and upcoming advancements in computer vision approaches have made processes such as invoice automation significantly accessible and robust.

About the author:

Prateek Sethi is a Data Scientist working at Mantra Labs. His work involves leveraging Artificial Intelligence to create data-driven solutions. Apart from his work he takes a keen interest in football and exploring the outdoors.

Further Reading:

Cancel

Knowledge thats worth delivered in your inbox

How Smarter Sales Apps Are Reinventing the Frontlines of Insurance Distribution

The insurance industry thrives on relationships—but it can only scale through efficiency, precision, and timely distribution. While much of the digital transformation buzz has focused on customer-facing portals, the real transformation is happening in the field, where modern sales apps are quietly driving a smarter, faster, and more empowered agent network.

Let’s explore how mobile-first sales enablement platforms are reshaping insurance sales across prospecting, onboarding, servicing, renewals, and growth.

The Insurance Agent Needs More Than a CRM

Today’s insurance agent is not just a policy seller—they’re also a financial advisor, data gatherer, service representative, and the face of the brand. Yet many still rely on paper forms, disconnected tools, and manual processes.

That’s where intelligent sales apps come in—not just to digitize, but to optimize, personalize, and future-proof the entire agent journey.

Real-World Use Cases: What Smart Sales Apps Are Solving

Across the insurance value chain, sales agent apps have evolved into full-service platforms—streamlining operations, boosting conversions, and empowering agents in the field. These tools aren’t optional anymore, they’re critical to how modern insurers perform. Here’s how leading insurers are empowering their agents through technology:

1. Intelligent Prospecting & Lead Management

Sales apps now empower agents to:

  • Prioritize leads using filters like policy type, value, or geography
  • Schedule follow-ups with integrated agent calendars
  • Utilize locators to look for nearby branch offices or partner physicians
  • Register and service new leads directly from mobile devices

Agents spend significantly less time navigating through disjointed systems or chasing down information. With quick access to prioritized leads, appointment scheduling, and location tools—all in one app—they can focus more on meaningful customer interactions and closing sales, rather than administrative overhead.

2. Seamless Policy Servicing, Renewals & Claims 

Sales apps centralize post-sale activities such as:

  • Tracking policy status, premium due date, and claims progress
  • Sending renewal reminders, greetings, and policy alerts in real-time
  • Accessing digital sales journeys and pre-filled forms.
  • Policy comparison, calculating premiums, and submitting documents digitally
  • Registering and monitoring customer complaints through the app itself

Customers receive a consistent and seamless experience across touchpoints—whether online, in-person, or via mobile. With digital forms, real-time policy updates, and instant access to servicing tools, agents can handle post-sale tasks like renewals and claims faster, without paperwork delays—leading to improved satisfaction and higher retention.

3. Remote Sales using Assisted Tools

Using smart tools, agents can:

  • Securely co-browse documents with customers through proposals
  • Share product visualizations in real time
  • Complete eKYC and onboarding remotely.

Agents can conduct secure, interactive consultations from anywhere—sharing proposals, visual aids, and completing eKYC remotely. This not only expands their reach to customers in digital-first or geographically dispersed markets, but also builds greater trust through real-time engagement, clear communication, and a personalized advisory experience—all without needing a physical presence.

4. Real-Time Training, Performance & Compliance Monitoring

Modern insurance apps provide:

  • On-demand access to training material
  • Commission dashboards and incentive monitoring
  • Performance reporting with actionable insights

Field agents gain access to real-time performance insights, training modules, and incentive tracking—directly within the app. This empowers them to upskill on the go, stay motivated through transparent goal-setting, and make informed decisions that align with overall business KPIs. The result is a more agile, knowledgeable, and performance-driven sales force.

5. End-to-End Sales Execution—Even Offline

Advanced insurance apps support:

  • Full application submission, from prospect to payment
  • Offline functionality in low-connectivity zones
  • Real-time needs analysis, quote generation, and e-signatures
  • Multi-login access with secure OTP-based authentication

Even in low-connectivity or remote Tier 2 and 3 markets, agents can operate at full capacity—thanks to offline capabilities, secure authentication, and end-to-end sales execution tools. This ensures uninterrupted productivity, faster policy issuance, and adherence to compliance standards, regardless of location or network availability.

6. AI-Powered Personalization for Health-Linked Products

Some forward-thinking insurers are combining AI with health platforms to:

  • Import real-time health data from fitness trackers or health apps 
  • Offer hyper-personalized insurance suggestions based on lifestyle
  • Enable field agents to tailor recommendations with more context

By integrating real-time health data from fitness trackers and wellness apps, insurers can offer hyper-personalized, preventive insurance products tailored to individual lifestyles. This empowers agents to move beyond transactional selling—becoming trusted advisors who recommend coverage based on customers’ health habits, life stages, and future needs, ultimately deepening engagement and improving long-term retention.

The Mantra Labs Advantage: Turning Strategy into Scalable Execution

We help insurers go beyond surface-level digitization to build intelligent, mobile-first ecosystems that optimize agent efficiency and customer engagement—backed by real-world impact.

Seamless Sales Enablement for Travel Insurance

We partnered with a leading travel insurance provider to develop a high-performance agent workflow platform featuring:

  • Secure Logins: Instant credential-based access without sign-up friction
  • Real-Time Performance Dashboards: At-a-glance insights into daily/monthly targets, policy issuance, and collections
  • Frictionless Policy Issuance: Complete issuance post-payment and document verification
  • OCR Integration: Auto-filled customer details directly from passport scans, minimizing errors and speeding up onboarding

This mobile-first solution empowered agents to close policies faster with significantly reduced paperwork and data entry time—improving agent productivity by 2x and enabling sales at scale.

Engagement + Analytics Transformation for Health Insurance

For one of India’s leading health insurers, we helped implement a full-funnel engagement and analytics stack:

  • User Journey Intelligence: Replaced legacy systems to track granular app behavior—policy purchases, renewals, claims, discounts, and drop-offs. Enabled real-time behavioral segmentation and personalized push/email notifications.
  • Gamified Wellness with Fitness Tracking: Added gamified fitness engagement, with rewards based on step counts and interactive nutrition quizzes—driving repeat app visits and user loyalty.
  • Attribution Tracking: Trace the exact source of traffic—whether it’s a paid campaign, referral program, or organic source—adding a layer of precision to marketing ROI.
  • Analytics: Integrated analytics to identify user interest segments. This allowed for hyper-targeted email and in-app notifications that aligned perfectly with user intent, driving both relevance and response rates.

Whether you’re digitizing field sales, gamifying customer wellness, or fine-tuning your marketing engine, Mantra Labs brings the technology depth, insurance expertise, and user-first design to turn strategy into scalable execution.

If you’re ready to modernize your agent network – Get in touch with us to explore how we can build intelligent, mobile-first tools tailored to your distribution strategy. Just remember, the best sales apps aren’t just tools, they’re growth engines; and field sales success isn’t about more apps. It’s about the right workflows, in the right hands, at the right time.

Cancel

Knowledge thats worth delivered in your inbox

Loading More Posts ...
Go Top
ml floating chatbot