10%

Try : Insurtech, Application Development

Edtech(5)

Events(34)

Interviews(10)

Life@mantra(11)

Logistics(1)

Strategy(14)

Testing(8)

Android(46)

Backend(29)

Dev Ops(3)

Enterprise Solution(22)

Frontend(28)

iOS(41)

Javascript(13)

Augmented Reality(17)

Customer Journey(12)

Design(13)

User Experience(34)

AI in Insurance(31)

Insurtech(59)

Product Innovation(37)

Solutions(15)

E-health(3)

HealthTech(8)

mHealth(3)

Telehealth Care(1)

Telemedicine(1)

Artificial Intelligence(109)

Bitcoin(7)

Blockchain(18)

Cognitive Computing(7)

Computer Vision(8)

Data Science(14)

FinTech(44)

Intelligent Automation(26)

Machine Learning(46)

Natural Language Processing(13)

How we used RetinaNet for dense shape detection in live imagery

Convolutional Neural Networks (CNN) have come a long way in conveniently identifying objects in images and videos. Networks like VGG19, ResNet, YOLO, SSD, R-CNN, DensepathNet, DualNet, Xception, Inception, PolyNet, MobileNet, and many more have evolved over time. Their range of applications lies in detecting space availability in a parking lot, satellite image analysis to track ships and agricultural output, radiology, people count, detecting words in vehicle license plates and storefronts, circuits/machinery fault analysis, medical diagnosis, etc.

Facebook AI Research (FAIR) has recently published RetinaNet architecture which uses Feature Pyramid Network (FPN) with ResNet. This architecture demonstrates higher accuracy in situations where speed is not really important. RetinaNet is built on top of FPN using ResNet.

Comparing tradeoff between speed and accuracy of different CNNs

Google offers benchmark comparison to calculate tradeoff between speed and accuracy of various networks using MS COCO dataset to train the models in TensorFlow. It gives us a benchmark to understand the best model that provides a balance between speed and accuracy. According to researchers, Faster R-CNN is more accurate, whereas R-FCN and FCN show better inference time (i.e. their speed is higher). Inception and ResNet are implementations of Faster R-CNN. MobileNet is an implementation of SSD.

Faster R-CNN implementations show an overall mAP (mean average precision) of around 30, which is highest for feature extraction. And, at the same time, its accuracy is also highest at around 80.5%. MobileNet R-FCN implementation has a lower mAP of around 15. Therefore, its accuracy drops down to about 71.5%. 

Thus, we can say — SSD implementations work best for detecting larger objects whereas, Faster R-CNN and R-FCN are better at detecting small objects.

speed and accuracy of various CNNs

On the COCO dataset, Faster R-CNN has average mAP for IoU (intersection-over-union) from 0.5 to 0.95 (mAP@[0.5, 0.95]) as 21.9% . R-FCN has mAP of 31.5% . SSD300 and SSD512 have mAPs of 23.2 and 26.8 respectively . YOLO-V2 is at 21.6% whereas YOLO-V3 is at 33% . FPN delivers 33.9% . RetinaNet stands highest at 40.8%.

RetinaNet- AP vs speed comparison
The two variations of RetinaNet are compared above for AP vs speed (ms) for inference.

One-stage detector vs two-stage detectors for shape detection

A One-stage detector scans for candidate objects sampled for around 100000 locations in the image that densely covers the spatial extent. This does not let the class balance between background and foreground. 

A Two-stage detector first narrows down the number of candidate objects on up to 2000 locations and separates them from the background in the first stage. It then classifies each candidate object in the second stage, thus managing the class balance. But, because of the smaller number of locations in the sample, many objects might escape detection. 

Faster R-CNN is an implementation of the two-stage detector. RetinaNet, an implementation of one stage detector addresses this class imbalance and efficiently detects all objects.

Focal Loss: a new loss function

This function focuses on training on hard negatives. It is defined as-

focal loss function

Where,

focal loss function

and p = sigmoid output score.

The greeks are hyperparameters.


When a sample classification is inappropriate and pₜ is small, it does not affect the loss. Gamma is a focusing parameter and adjusts the rate at which the easy samples are down-weighted. Samples get down-weighted when their classification is inappropriate and pₜ is close to 1. When gamma is 0, the focal loss is close to the cross-entropy loss. Upon increasing gamma, the effect of modulating factor also increases.

RetinaNet Backbone

The new loss function called Focal loss increases the accuracy significantly. Essentially it is a one-stage detector Feature Pyramid Network with Focal loss replacing the cross-entropy loss. 

Hard negative mining in a single shot detector and Faster R-CNN addresses the class imbalance by downsampling the dominant samples. On the contrary, RetinaNet addresses it by changing the weights in the loss function. The following diagram explains the architecture.

RetinaNet architecture

Here, deep feature extraction uses ResNet. Using FPN on top of ResNet further helps in constructing a multi-scale feature pyramid from a single resolution image. FPN is fast to compute and works efficiently on multiscale.

Results

We used ResNet50-FPN pre-trained on MS COCO to identify humans in the photo. The threshold is set above a score of 0.5. The following images show the result with markings and confidence values.

Dense shape detection
Human shape detection

We further tried to detect other objects like chairs.

RetinaNet object detection

Conclusion: It’s great to know that training on the COCO dataset can detect objects from unknown scenes. The object detection in the scenes took 5-7 seconds. So far, we have put filters of human or chair in results. RetinaNet can detect all the identifiable objects in the scene.

Multiple objects detection using RetinaNet

The different objects detected with their score are listed below-

human0.74903154
human0.7123633
laptop0.69287986
human0.68936586
bottle0.67716646
human0.66410005
human0.5968385
chair0.5855772
human0.5802317
bottle0.5792091
chair0.5783555
chair0.538948
human0.52267283

Next, we will be interested in working on a model good in detecting objects in the larger depth of the image, which the current ResNet50-FPN could not do.

About author: Harsh Vardhan is a Tech Lead in the Development Department of Mantra Labs. He is integral to AI-based development and deployment of projects at Mantra Labs.

General FAQs

What is RetinaNet?

RetinaNet is a type of CNN (Convolutional Neural Network) architecture published by Facebook AI Research also known as FAIR. It uses the Feature Pyramid Network (FPN) with ResNet. RetinaNet is widely used for detecting objects in live imagery (real-time monitoring systems). This architecture demonstrates a high-level of accuracy, but with a little compromise in speed. In the experiment we conducted, it took 5-7 seconds for object detection in live scenes.Dense shape detection - RetinaNet

What is RetinaNet Model?

RetinaNet model comprises of a backbone network and two task-specific sub-networks. The backbone network is a Feature Pyramid Network (FPN) built on ResNet. It is responsible for computing a convolution feature (object) from the input imagery. The two subnetworks are responsible for the classification and box regression, i.e. one subnet predicts the possibility of the object being present at a particular spatial location and the other subnetwork outputs the object location for the anchor box.

What is Focal Loss?

The focal loss function focuses on training on hard negatives. In other words, the focal loss function is an algorithm for improving Average Precision (AP) in single-stage object detectors. It is defined as-RetinaNet focal loss function

What is SSD Network?

Single Shot Detector (SSD) can detect multiple objects in an image in a single shot, hence the name. 
The beauty of SSD networks is that it predicts the boundaries itself and has no assigned region proposal network. SSD networks can predict the boundary boxes and classes from feature maps in just one pass by using small convolutional filters.

Glossary of Terms related to convolutional neural networks

CNN

Deep Learning uses Convolutional neural networks (CNN) for analyzing visual imagery. It consists of an input and output layer and multiple intermediate layers. In CNN programming, the input is called a tensor, which is usually an image or a video frame. It passes through the convolutional layer forming an abstract feature map identifying different shapes.

R-CNN

The process of combining region proposals with CNN is called as R-CNN. Region proposals are the smaller parts of the original image that have a probability of containing the desired shape/object. The R-CNN algorithm creates several region proposals and each of them goes to the CNN network for better dense shape detection.

ResNet

Residual Neural Network (ResNet) utilizes skip connections to jump over some layers. Classical CNNs do not perform when the depth of the network increases beyond a certain threshold. Most of the ResNet models are implemented with double or triple layer skips with batch normalization in between. ResNet helps in the training of deeper networks.

YOLO

You only look once (YOLO) is a real-time object detection system. It is faster than most other neural networks for detecting shapes and objects. Unlike other systems, it applies neural network functions to the entire image, optimizing the detection performance.

FAIR

It is Facebook’s AI Research arm for understanding the nature of intelligence and creating intelligent machines. The main research areas at FAIR include Computer Vision, Conversational AI, Integrity, Natural Language Processing, Ranking and Recommendations, System Research, Theory, Speech & Audio, and Human & Machine Intelligence.

FPN

Feature Pyramid Network (FPN) is a feature extractor designed for achieving speed and accuracy in detecting objects or shapes. It generates multiple feature map layers with better quality information for object detection.

COCO Dataset

Common Objects in Context (COCO) is a large-scale dataset for detecting, segmenting, and captioning any object. 

FCN

Fully Convolutional Network (FCN) transforms the height and width of the intermediate layer (feature map) back to the original size so that predictions have a one-to-one correspondence with the input image. 

R-FCN

R-FCN corresponds to a region-based fully convolutional network. It is mainly used for feature detection. R-FCN comprises region-based feature maps that are independent of region proposals (ROI) and carry computation outside of ROIs. It is much simpler and about 20 times faster than R-CNN. 

TensorFlow

It is an open-source software library developed by Google Brain for a range of dataflow and differential programming applications. It is also useful in neural network programming. 

Also read – How are Medical Images shared among Healthcare Enterprises



Cancel

Knowledge thats worth delivered in your inbox

12 Tips To Secure Your Mobile Application

Cyber attacks and data theft have become so common these days especially when it comes to mobile applications. As a result, mobile apps that experience security breaches may suffer financial losses. With many hackers eyeing to steal customer data, securing these applications has become the number one priority for organizations and a serious challenge for developers. According to Gartner’s recent research, Hype Cycle for Application Security, investment in application security will increase by more than two-fold over the next few years, from $6 billion this year to $13.7 billion by 2026. Further, the report stated, “Application security is now top-of-mind for developers and security professionals, and the emphasis is now turning to apps hosted in public clouds,” It is crucial to get the fundamental components of DevOps security correct. Here are the 12 tips to secure your mobile application: 

1. Install apps from trusted sources:

It’s common to have Android applications republished on alternate markets or their APKs & IPAs made available for download. Both APK and IPA may be downloaded and installed from a variety of places, including websites, cloud services, drives, social media, and social networking. Only the Play Store and the App Store should be allowed to install trustworthy APK and IPA files. To prevent utilizing these apps, we should have a source check detection (Play Store or App Store) upon app start.

Also read, https://andresand.medium.com/add-method-to-check-which-app-store-the-android-app-is-installed-from-or-if-its-sideloaded-c9f450a3d069

2. Root Detection:

Android: An attacker could launch a mobile application on a rooted device and access the local memory or call specific activities or intents to perform malicious activities in the application. 

iOS: Applications on a jailbroken device run as root outside of the iOS sandbox. This can allow applications to access sensitive data stored in other apps or install malicious software negating sandboxing functionality. 

More on Root Detection- https://owasp.org/www-project-mobile-top-10/2016-risks/m8-code-tampering

3. Data Storing:

Developers use Shared Preferences & User Defaults to store key-value pairs like tokens, mobile numbers, email, boolean values, etc. Additionally, while creating apps, developers prefer SQLite databases for structured data. It is recommended to store any data in the format of encryption so that it is difficult to extract the information by hackers.

4. Secure Secret Keys:

API keys, passwords, and tokens shouldn’t be hardcoded in the code. It is recommended to use different techniques to store these values so that hackers cannot get away quickly by tampering with the application. 

Here’s a reference link: https://guides.codepath.com/android/Storing-Secret-Keys-in-Android

5. Code Obfuscation

An attacker may decompile the APK file and extract the source code of the application. This may expose sensitive information stored in the source code of the application to the attacker which may be used to perform tailored attacks. 

It is better to obfuscate the source code to prevent all the sensitive information contained in the source code.

6. Secure Communication:

An attacker may perform malicious activities to leverage the level of attacks since all communication is happening over unencrypted channels. So always use HTTPS URLs over HTTP URLs.

7. SSL Pinning:

Certificate pinning allows mobile applications to restrict communication only to servers with a valid certificate matching the expected value (pin). Pinning ensures that no network data is compromised even if a user is tricked into installing a malicious root certificate on their mobile device. Any app that pins its certificates would thwart such phishing attempts by refusing to transmit data over a compromised connection

Please refer: 

https://owasp.org/www-community/controls/Certificate_and_Public_Key_Pinning

8. Secure API request & response data

The standard practice is to use HTTPS for the baseline protection of REST API calls. The information sent to the server or received from the server may be further encrypted with AES, etc. For example, if there are sensitive contents, you might choose to select those to encrypt so that even if the HTTPS is somehow broken or misconfigured, you have another layer of protection from your encryption.

9. Secure Mobile App Authentication:

In case an application does not assign distinct and complex session tokens after login to a user, an attacker can conduct phishing in order to lure the victim to use a custom-generated token provided by the attacker and easily bypass the login page with the captured session by using a MiTM attack.

i) Assign a distinct and complex session token to a user each time he/she logs on successfully to the application. 

ii) Terminate the session lifetime immediately after logging out. 

iii) Do not use the same session token for two or more IP addresses. 

iv) Limit the expiry time for every session token.

10.  Allow Backup 

Disallow users to back up an app if it contains sensitive data. Having access to backup files (i.e. when android:allowBackup=”true”), it is possible to modify/read the content of an app even on a non-rooted device. So it is recommended to make allow backup false. 

11. Restrict accessing android application screens from other apps

Ideally, your activities should not give any provision to the opening from other services or applications. Make it true only when you have a specific requirement to access your flutter screens from other apps otherwise change to android:exported= ”false”

12. Restrict installing packages from the android application

REQUEST_INSTALL_PACKAGES permission allows apps to install new packages on a user’s device. We are committed to preventing abuse on the Android platform and protecting users from apps that self-update using any method other than Google Play’s update mechanism or download harmful APKs.

Conclusion: 

Mobile Apps have become more personalized than ever before with heaps of customers’ personal data stored in them every day. In order to build trust and loyalty among users and prevent significant financial and credential losses for the companies, it is now crucial to make sure the application is secure for the user. Following the above-mentioned mobile app security checklists will definitely help to prevent hackers from hacking the app.

About the Author:

Raviteja Aketi is a Senior Software Engineer at Mantra Labs. He has extensive experience with B2B projects. Raviteja loves exploring new technologies, watching movies, and spending time with family and friends.

Read our latest blog: Implementing a Clean Architecture with Nest.JS

Cancel

Knowledge thats worth delivered in your inbox

Loading More Posts ...