ABot-OCR Technical Report

2026-05-27

AI Summary

This work addresses the limitations of traditional document image transcription systems, which rely on multi-stage pipelines and exhibit fragile structure parsing, by introducing the first end-to-end vision-language model capable of directly transcribing full-page document images into structurally well-formed Markdown. The core innovations include a novel reinforcement learning approach with structural constraints—termed Decoupled Heterogeneous Document Optimization—a large-scale synthetic data engine designed to ensure structural consistency, and a joint optimization framework that integrates supervised fine-tuning with format compliance. The proposed method achieves state-of-the-art performance on OmniDocBench v1.5 and v1.6 with scores of 92.81 and 93.30, respectively, and demonstrates exceptional multilingual generalization across ten languages.

Abstract

We introduce ABot-OCR, an end-to-end vision-language model that transcribes a page image directly into clean Markdown in a single forward pass. By doing so, our approach completely eliminates the need for brittle modular orchestration. To maximize parsing fidelity, we develop a dedicated data engine to provide large-scale, structurally consistent supervision. Furthermore, we propose Decoupled Heterogeneous Document Optimization, a structure-constrained reinforcement learning method that sharpens textual accuracy and strictly enforces markup well-formedness beyond supervised fine-tuning alone. Extensive evaluations demonstrate the superior performance of our framework. On the OmniDocBench v1.5 and v1.6 benchmarks, ABot-OCR achieves state-of-the-art scores of 92.81 and 93.30 among all end-to-end systems, substantially narrowing the performance gap relative to strong pipeline baselines. Finally, comprehensive multilingual text recognition across ten diverse languages further confirms the robust generalizability of ABot-OCR.

Problem

• document image transcription
• end-to-end OCR
• Markdown generation
• vision-language model
• structured output

Innovation

• end-to-end OCR
• vision-language model
• structured Markdown generation
• reinforcement learning with structural constraints
• large-scale data engine