DocR1: Evidence Page-Guided GRPO for Multi-Page Document Understanding

2025-08-10

AI Summary

Multi-page document understanding demands fine-grained cross-page visual perception and multi-hop reasoning—capabilities inadequately supported by current multimodal large language models (MLLMs). To address this, we propose EviGRPO, a framework guided by evidence pages that integrates an evidence-aware reward mechanism and a two-stage annotation pipeline, progressively optimizing from coarse-grained page localization to fine-grained reasoning via curriculum learning. We further introduce EviBench, a high-quality training dataset, and ArxivFullQA—the first long-document question answering benchmark featuring full-length academic papers. Extensive experiments demonstrate that EviGRPO achieves state-of-the-art performance across multiple multi-page document benchmarks while maintaining competitive accuracy on single-page tasks, validating its generality and effectiveness.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Understanding multi-page documents poses a significant challenge for multimodal large language models (MLLMs), as it requires fine-grained visual comprehension and multi-hop reasoning across pages. While prior work has explored reinforcement learning (RL) for enhancing advanced reasoning in MLLMs, its application to multi-page document understanding remains underexplored. In this paper, we introduce DocR1, an MLLM trained with a novel RL framework, Evidence Page-Guided GRPO (EviGRPO). EviGRPO incorporates an evidence-aware reward mechanism that promotes a coarse-to-fine reasoning strategy, guiding the model to first retrieve relevant pages before generating answers. This training paradigm enables us to build high-quality models with limited supervision. To support this, we design a two-stage annotation pipeline and a curriculum learning strategy, based on which we construct two datasets: EviBench, a high-quality training set with 4.8k examples, and ArxivFullQA, an evaluation benchmark with 8.6k QA pairs based on scientific papers. Extensive experiments across a wide range of benchmarks demonstrate that DocR1 achieves state-of-the-art performance on multi-page tasks, while consistently maintaining strong results on single-page benchmarks.

Problem

• Enhancing multi-page document understanding with MLLMs
• Developing evidence-aware RL for coarse-to-fine reasoning
• Creating datasets for training and evaluating document QA

Innovation

• Evidence Page-Guided GRPO for document understanding
• Evidence-aware reward for coarse-to-fine reasoning
• Two-stage annotation and curriculum learning strategy