MonkeyOCRv2: A Visual-Text Foundation Model for Document AI
This work addresses the limitations of mainstream vision encoders—pretrained on natural images—in capturing character-level visual details and layout structures essential for document understanding. To bridge this gap, the authors propose a document-oriented vision-language pretraining approach, leveraging a newly constructed dataset, MonkeyDoc v2, comprising 113 million multilingual document images. Their method jointly optimizes image-to-text generation and pixel-level document reconstruction to effectively align visual and textual representations. As the first large-scale foundational model for documents, it employs a frozen vision encoder paired with a lightweight 0.7B-parameter language model, outperforming even 3B-parameter models by 2.8% on MDPBench and achieving state-of-the-art results across eight comprehension benchmarks and five categories of document analysis tasks, surpassing existing methods based on CLIP, DINO, and SAM.