Zero-Shot Synthetic-to-Real Handwritten Text Recognition via Task Analogies

2026-04-08

AI Summary

This work addresses the poor generalization of synthetic handwritten text recognition models in the absence of real target-language data by proposing a zero-shot cross-lingual domain adaptation method. The approach models the parameter correction patterns from synthetic to real handwritten text in source languages and transfers these corrections to the target language through a language-similarity-weighted fusion of multi-source transfer knowledge, enabling cross-domain generalization without any real target-domain data. Experimental results demonstrate that the proposed method significantly outperforms synthetic-data-only baselines across five languages and six mainstream architectures. Notably, it achieves the first effective zero-shot synthetic-to-real handwritten text recognition and remains effective even for languages belonging to different language families.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Application Domains: Humanities & Computational Social Science

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Handwritten Text Recognition (HTR) models trained on synthetic handwriting often struggle to generalize to real text, and existing adaptation methods still require real samples from the target domain. In this work, we tackle the fully zero-shot synthetic-to-real generalization setting, where no real data from the target language is available. Our approach learns how model parameters change when moving from synthetic to real handwriting in one or more source languages and transfers this learned correction to new target languages. When using multiple sources, we rely on linguistic similarity to weigh their contrubition when combining them. Experiments across five languages and six architectures show consistent improvements over synthetic-only baselines and reveal that the transferred corrections benefit even languages unrelated to the sources.

Problem

• Handwritten Text Recognition
• Zero-Shot
• Synthetic-to-Real
• Domain Generalization
• Cross-Lingual Transfer

Innovation

• zero-shot adaptation
• synthetic-to-real generalization
• handwritten text recognition
• parameter correction transfer
• cross-lingual transfer