TEACH: Text Encoding as Curriculum Hints for Scene Text Recognition

2025-08-01

AI Summary

Scene text recognition (STR) suffers from insufficient robustness due to severe visual distortions and weak semantic priors. To address this, we propose TEACH, a novel training paradigm that encodes textual labels as curriculum-based prompts in an embedding space and progressively masks label inputs via a loss-aware masking mechanism—enabling a smooth transition from label-assisted to purely vision-based recognition. TEACH requires no external pretraining, introduces zero inference overhead, and is model-agnostic and plug-and-play. Evaluated on multiple benchmarks, it achieves significant accuracy improvements, particularly under challenging conditions such as low-quality images and complex fonts. The method demonstrates strong generalization and stability, effectively expanding the capability frontier of vision-only STR systems.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Computer Vision: Language and Vision
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Scene Text Recognition (STR) remains a challenging task due to complex visual appearances and limited semantic priors. We propose TEACH, a novel training paradigm that injects ground-truth text into the model as auxiliary input and progressively reduces its influence during training. By encoding target labels into the embedding space and applying loss-aware masking, TEACH simulates a curriculum learning process that guides the model from label-dependent learning to fully visual recognition. Unlike language model-based approaches, TEACH requires no external pretraining and introduces no inference overhead. It is model-agnostic and can be seamlessly integrated into existing encoder-decoder frameworks. Extensive experiments across multiple public benchmarks show that models trained with TEACH achieve consistently improved accuracy, especially under challenging conditions, validating its robustness and general applicability.

Problem

• STR struggles with complex visuals and limited semantics
• TEACH injects ground-truth text as curriculum guidance
• Enables label-to-visual transition without external pretraining

Innovation

• Inject ground-truth text as auxiliary input
• Progressively reduce label influence during training
• Model-agnostic, no external pretraining needed