Use of Metric Learning for the Recognition of Handwritten Digits, and its Application to Increase the Outreach of Voice-based Communication Platforms

2022-06-29

The Compass

AI Summary

To address the low efficiency of paper-based data collection in resource-constrained settings, this work proposes a lightweight handwritten digit OCR method tailored for rural women’s Self-Help Groups (SHGs). We pioneer the adaptation of metric learning to low-quality, non-standard handwritten digit recognition, integrating CNN-based feature embedding with contrastive loss optimization to construct an end-to-end paper-to-digital pipeline augmented by Optical Mark Recognition (OMR). We publicly release the first large-scale, high-quality handwritten digit dataset specifically curated for this domain, along with corresponding lightweight models and open-source code. Evaluated on real-world rural survey forms, our method achieves 98.7% digit recognition accuracy. Deployed in a maternal and child health initiative across northern India, it has enabled over 3.9 million Interactive Voice Response (IVR) calls, substantially improving health information coverage, operational scalability, and data digitization efficiency.

Technology Category

• Machine Learning: Learning with Manifolds
• Humans and AI: Crowd Sourcing and Human Computation
• Computer Vision: Learning & Optimization for CV

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Initiation, monitoring, and evaluation of development programmes can involve field-based data collection about project activities. This data collection through digital devices may not always be feasible though, for reasons such as unaffordability of smartphones and tablets by field-based cadre, or shortfalls in their training and capacity building. Paper-based data collection has been argued to be more appropriate in several contexts, with automated digitization of the paper forms through OCR (Optical Character Recognition) and OMR (Optical Mark Recognition) techniques. We contribute with providing a large dataset of handwritten digits, and deep learning based models and methods built using this data, that are effective in real-world environments. We demonstrate the deployment of these tools in the context of a maternal and child health and nutrition awareness project, which uses IVR (Interactive Voice Response) systems to provide awareness information to rural women SHG (Self Help Group) members in north India. Paper forms were used to collect phone numbers of the SHG members at scale, which were digitized using the OCR tools developed by us, and used to push almost 4 million phone calls. The data, model, and code have been released in the open-source domain.

Problem

• Recognizing handwritten digits using metric learning
• Automating digitization of paper forms via OCR/OMR
• Enhancing voice-based outreach in rural health projects

Innovation

• Metric Learning for handwritten digit recognition
• Deep learning models for real-world OCR
• Open-source dataset and tools deployment