PLATTER: A Page-Level Handwritten Text Recognition System for Indic Scripts

2025-02-10

AI Summary

To address the lack of cross-lingual evaluation, insufficient support for Indic scripts, and neglect of handwritten text detection (HTD) in handwritten text recognition (HTR), this paper introduces the first page-level end-to-end handwritten OCR framework tailored for Indic scripts. Methodologically, it employs an enhanced YOLOv8 for language-agnostic handwritten text detection and integrates multiple CRNN and Transformer-based architectures for multilingual recognition, augmented by synthetic data generation and advanced augmentation to mitigate annotation scarcity. Key contributions include: (1) releasing CHIPS—the first large-scale, fully annotated, page-level Indic handwritten OCR dataset, covering 10 languages and thousands of real-world document pages; (2) establishing a unified, cross-model and cross-lingual evaluation benchmark; and (3) open-sourcing the complete codebase, pre-trained models, and evaluation toolchain. Experiments conduct standardized comparisons of six HTR models across all 10 Indic languages, significantly advancing low-resource handwritten OCR research.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Language and Vision

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

In recent years, the field of Handwritten Text Recognition (HTR) has seen the emergence of various new models, each claiming to perform competitively better than the other in specific scenarios. However, making a fair comparison of these models is challenging due to inconsistent choices and diversity in test sets. Furthermore, recent advancements in HTR often fail to account for the diverse languages, especially Indic languages, likely due to the scarcity of relevant labeled datasets. Moreover, much of the previous work has focused primarily on character-level or word-level recognition, overlooking the crucial stage of Handwritten Text Detection (HTD) necessary for building a page-level end-to-end handwritten OCR pipeline. Through our paper, we address these gaps by making three pivotal contributions. Firstly, we present an end-to-end framework for Page-Level hAndwriTTen TExt Recognition (PLATTER) by treating it as a two-stage problem involving word-level HTD followed by HTR. This approach enables us to identify, assess, and address challenges in each stage independently. Secondly, we demonstrate the usage of PLATTER to measure the performance of our language-agnostic HTD model and present a consistent comparison of six trained HTR models on ten diverse Indic languages thereby encouraging consistent comparisons. Finally, we also release a Corpus of Handwritten Indic Scripts (CHIPS), a meticulously curated, page-level Indic handwritten OCR dataset labeled for both detection and recognition purposes. Additionally, we release our code and trained models, to encourage further contributions in this direction.

Problem

• Develops end-to-end page-level handwritten text recognition
• Addresses lack of Indic language datasets in HTR
• Enables consistent comparison of HTR models

Innovation

• Page-level handwritten text recognition
• Language-agnostic HTD model
• Corpus of Handwritten Indic Scripts