Out of Length Text Recognition with Sub-String Matching

2024-07-17

arXiv.org

AI Summary

Existing text recognition models trained solely on short, word-level annotations struggle to generalize to arbitrarily long scene text—a challenge we term “Out-of-Length” (OOL) recognition. Method: We introduce the first long-text benchmark, LTB, and propose SMTR, an iterative cross-attention framework based on substring matching. SMTR jointly encodes substrings, performs substring matching, applies discrepancy-aware regularization, and incorporates inference-time substring disambiguation—enabling long-range dependency modeling and structural consistency learning without requiring long-text annotations. Contribution/Results: SMTR achieves state-of-the-art performance on standard short-text benchmarks (e.g., IIIT5K, SVT) and our new LTB benchmark, demonstrating the first robust long-text recognition capability trained exclusively on short-text data. This establishes a novel low-resource paradigm for scene text recognition, eliminating the need for costly long-sequence labeling while preserving accuracy and generalization.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Language and Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Scene Text Recognition (STR) methods have demonstrated robust performance in word-level text recognition. However, in real applications the text image is sometimes long due to detected with multiple horizontal words. It triggers the requirement to build long text recognition models from readily available short (i.e., word-level) text datasets, which has been less studied previously. In this paper, we term this task Out of Length (OOL) text recognition. We establish the first Long Text Benchmark (LTB) to facilitate the assessment of different methods in long text recognition. Meanwhile, we propose a novel method called OOL Text Recognition with sub-String Matching (SMTR). SMTR comprises two cross-attention-based modules: one encodes a sub-string containing multiple characters into next and previous queries, and the other employs the queries to attend to the image features, matching the sub-string and simultaneously recognizing its next and previous character. SMTR can recognize text of arbitrary length by iterating the process above. To avoid being trapped in recognizing highly similar sub-strings, we introduce a regularization training to compel SMTR to effectively discover subtle differences between similar sub-strings for precise matching. In addition, we propose an inference augmentation strategy to alleviate confusion caused by identical sub-strings in the same text and improve the overall recognition efficiency. Extensive experimental results reveal that SMTR, even when trained exclusively on short text, outperforms existing methods in public short text benchmarks and exhibits a clear advantage on LTB. Code: https://github.com/Topdu/OpenOCR.

Problem

• Long Sentence Recognition
• Short Word Data
• Text Identification

Innovation

• Substring Matching Text Recognition (SMTR)
• Attention Mechanisms
• Out-of-length Text Recognition Benchmark