Try : Insurtech, Application Development










Dev Ops(2)

Enterprise Solution(20)




AI in Insurance(26)


Product Innovation(34)


Augmented Reality(7)

Customer Journey(7)


User Experience(21)

Artificial Intelligence(95)



Cognitive Computing(7)

Computer Vision(6)

Data Science(14)


Intelligent Automation(25)

Machine Learning(43)

Natural Language Processing(10)




Telehealth Care(1)

Reimagining Medical Diagnosis with Chatbots

4 minutes, 51 seconds read

Chatbots are rapidly gaining popularity in the healthcare sector. According to research conducted by Grand View Research, the global chatbot market is expected to reach $1.23 billion by 2025 growing at a CAGR of 24.3%. The current COVID pandemic has caused a lot of stress in the healthcare sector, with hospitals getting swamped with COVID-19 patients and also handling regular consults. 

This has made medical chatbots very attractive, helping in scheduling appointments, custom support, symptom checks, providing nutrition and wellness information, mental therapy, etc. Let’s take a look at how chatbots are transforming the digital transformation in the healthcare sector.

The shift to Medical Chatbots and Telemedicine

Lockdowns and social distancing due to COVID-19 gave a significant boost to digital business models. Organizations had to find ways to keep up the operations, make business continuity plans, and engage the workforce working remotely. Even healthcare providers took to technology such as telemedicine, chatbots, and remote monitoring equipment for patients who were not able to visit doctors in person. 

Many hospitals had been trying to implement telemedicine over the last couple of years, at least for ailments that can do without in-person diagnosis and can be cured by prescribing medicines based on symptoms told by the patient. COVID-19 gave that extra push for telemedicine. 

Another tendency that people have these days is to search for information on Google for self-diagnosis. However, that may not be effective. Therefore, many people are turning towards healthcare chatbots for medical information. 

Multilingual AI chatbot with video for diagnostic services – Hitee.chat

The Role of Chatbots in Medical Diagnosis 

The entire experience from admission to discharge is one of the key differentiators for patients while choosing a healthcare provider. People want quicker services and instant answers to their queries. 

With the coronavirus outbreak, hospitals and clinics are facing additional pressure. It has created a dire need for technology such as medical chatbots to provide better patient experience. 

Currently, there are some chatbots that leverage AI and machine learning to provide diagnoses by using algorithms to run the responses through a database of medical literature available. Let’s take a look at possible situations where chatbots play a crucial role in diagnostics-

  • Reliability: Instead of using a search engine to find answers, people will find chatbots more reliable for medical information. They need to be backed by legitimate medical databases to provide better accuracy.
  • Medical History: Chatbots cannot replace the role of a doctor while diagnosing but it can be of great assistance to them in providing medical history to better diagnose the health issue.
  • Triggering Attention: There are many symptom checking apps and bots available today which are widely used to check symptoms for possible diseases. Even with the nearest possible result in hand, it triggers the patient to a doctors’ visit if the symptoms seem grave. 
  • Support for Healthcare Workers: In case of mild diseases such as common cold, indigestion, minor wounds, etc. Chatbots are of great help as they reduce the workload of health workers who can focus on critical patients. 
  • Ensure Confidentiality: In some cases, patients may not be comfortable to open up to a doctor in person, but finds it easier to answer questions by a chatbot. Especially, when it comes to mental illness. 
  • Availability: Although rare, but there can be cases when medical help is not available physically such as during curfews or lockdowns. In such situations chatbots can be of great help for immediate medical support. 

Prevailing Challenges

Chatbots can provide basic medical information or do a cursory diagnosis of a health problem. However, the biggest challenge with diagnostic chatbots is the accuracy of the output. 

Research by the National Center for Biotechnology Information (NCBI) suggests that computer-based diagnostic support tools can be very beneficial to clinicians. But the effectiveness of 23 symptom checkers reported deficits and only 34% of standard patient evaluations were achieved in the first attempt. 

Unlike actual doctors, chatbots cannot feel the pulse, check the heartbeat or blood pressure, check the body part where the issue is, etc. Patients these days tend to self-diagnose quite often but they may not understand the diagnoses. 

Medical Chatbots can provide the information but can they explain it like a doctor as well? That would be debatable. Not everyone can understand medical jargon. Another issue is the risk of error in diagnosis. Too much dependency on the diagnosis can have steep consequences putting lives at risk. 

Redefining Chatbots in Medical Diagnosis

Currently, the chatbots function primarily through text while chatting with the patient. But in the coming future, it has a huge scope of improvement when combined with videos, images, voice recognition it will provide better information to the chatbot to provide better diagnoses. 

Medical diagnosis chatbot with video – Hitee.chat

Technologies like Natural Language Processing (NLP), machine learning, AI algorithms will enable better processing of the data and help clinicians with quicker diagnosis. It is possible to increase the capability of these chatbots through broader data and technologies. NLP integrated chatbots can also cater to specially-abled patients. 

More usage of diagnostic chatbots will make people take better care of their health. Indeed, there is scope for improvement for chatbots in medical diagnosis. But at the same time, reliability on them is also gradually increasing.

Down the Road

Chatbots in medical diagnosis can act as an aid to clinicians, reduce workload for healthcare workers, provide instant answers, and in some cases, it is a cheaper medium and lesser hassle than to visit a hospital. 

Bots have huge potential to streamline diagnosis. It won’t be a surprise to see chatbots be the first point of contact for medical help. 

We’ve introduced a multilingual AI-powered video chatbot for hospitals, private clinics, and diagnostic services. It can automate appointment bookings, checking symptoms, provide information, answer FAQs and more. You can write to us at hello@mantralabsglobal.com for your specific requirements.

Website: Hitee.chat

To know more about how HealthTech is reshaping the healthcare industry in bringing hospitals to a customer’s doorstep, watch our webinar on Digital Health Beyond COVID-19


Knowledge thats worth delivered in your inbox

Tabular Data Extraction from Invoice Documents

5 minutes, 12 seconds read

The task of extracting information from tables is a long-running problem statement in the world of machine learning and image processing. Although the latest accomplishments in the field of deep learning have seen a lot of success, tabular data extraction still remains a challenge due to the vast amount of ways in which tables are represented both visually and structurally. Below are some of the examples: 

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Invoice Documents

Many companies process their bills in the form of invoices which contain tables that hold information about the items along with their prices and quantities. This information is generally required to be stored in databases while these invoices get processed.

Traditionally, this information is required to be hand filled into a database software however, this approach has some drawbacks:

1. The whole process is time consuming.

2. Certain errors might get induced during the data entry process.

3. Extra cost of manual data entry.

 An invoice automation system can be deployed to address these shortcomings. The idea is to upload the invoice document and the system will read and generate the tabular information in the digital format making the whole process faster and more cost-effective for companies.

Fig. 6

Fig. 6 shows a sample invoice that contains some regular invoice details such as Invoice No, Invoice Date, Company details, and two tables holding transaction information. Now, our goal is to extract the information present in the two tables.

Tabular Information

The problem of extracting tables from invoices can be condensed into 2 main subtasks.

1. Table Detection

2. Tabular Structure Extraction.

 What is Table Detection?

 Table Detection is the process of identifying and locating tables that are present in a document, usually an image. There are multiple ways to detect tables in an image. Some of the approaches make use of image processing toolkits like OpenCV while some of the other approaches use statistical models on features extracted from the documents such as Text Position and Text Characteristics. Recently more deep learning approaches have been used to detect tables using trained neural networks similar to the ones used in Object Detection.

What is Table Structure Extraction?

Table Structure Extraction is the process of extracting the tabular information once the boundaries of the table are detected through Table Detection. The information within the rows and columns is then extracted and transferred to the desired format, usually CSV or Excel file.

Table Detection using Faster RCNN

Faster RCNN is a neural network model that comes from the RCNN family. It is the successor of Fast RCNN created by Ross Girshick in 2015. The name Faster RCNN is to signify an improvement over the previous model both in terms of training speed and detection speed. 

To read more about the model framework, one can access the paper Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.

 There are many other object detection model architectures that are available for use today. Each model comes with certain advantages and disadvantages in terms of prediction accuracy, model parameter size, inference speed, etc.

For the task of detecting tables in invoice documents, we will select the Faster RCNN model with FPN(Feature Pyramid Network) as a feature extraction network. The model is pre-trained on the ImageNet corpus using ResNET 101 architecture. The ImageNet corpus is a public dataset that consists of more than 20,000 image categories of everyday objects.  We will therefore make use of a Pytorch framework to train and test the model.

The above mentioned model gives us a fast inference time and a high Mean Average Precision. It is preferred for cases where a quick real time detection is desired.

First, the model is to be trained using public datasets for Table Detection such as Marmot and UNLV datasets. Next, we further fine-tune the model with our custom labeled dataset. For the purpose of labeling, we will follow the COCO annotation format.

Once trained, the model displayed an accuracy close to 86% on our custom dataset. There are certain scenarios where the model fails to locate the tables such as cases containing watermarks and/or overlapping texts. Tables without borders are also missed in a few instances. However, the model has shown its ability to learn from examples and detect tables in multiple different invoice documents. 

Fig. 7

After running inference on the sample invoice from Fig 6, we can see two table boundaries being detected by the model in Fig 7. The first table gets detected with 100% accuracy and the second table is detected with 99% accuracy.

Table Structure Extraction

Once the boundaries of the table are detected by the model, an OCR (Optical Character Reader) mechanism is used to extract the text within the boundaries. The text is then processed using the information that is part of a unique table.

We were able to extract the correct structure of the table, including its headers and line items using logics derived from the invoices. The difficulty of this process depends on the type of invoice format at hand.

There are multiple challenges that one may encounter while building an algorithm to extract structure. Some of them are:

  1. The span of some table columns may overlap making it difficult to determine the boundaries between columns.
  2. The fonts and sizes present within tables may vary from one table to another. The algorithm should be able to accomodate for this variation.
  3. The tables might get split into two pages and detecting the continuation of a table might be challenging.

Certain deep learning approaches have also been published recently to determine the structure of a table. However, training them on custom datasets still remains a challenge. 

Fig 8

The final result is then stored in a CSV file and can be edited or stored according to one’s convenience as shown in Fig 8 which displays the first table information.


The deep learning approach to extracting information from structured documents is a step in the right direction. With high accuracy and low running time, the systems can only learn to perform better with more data. The recent and upcoming advancements in computer vision approaches have made processes such as invoice automation significantly accessible and robust.

About the author:

Prateek Sethi is a Data Scientist working at Mantra Labs. His work involves leveraging Artificial Intelligence to create data-driven solutions. Apart from his work he takes a keen interest in football and exploring the outdoors.

Further Reading:


Knowledge thats worth delivered in your inbox

Loading More Posts ...
Go Top

May i help you?

bot shadow

Our Website is
Best Experienced on
Chrome & Safari

safari icon