Doc-V*:Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA

2026-04-15

AI Summary

This work addresses the challenge of long-form visual question answering over multi-page documents, which demands intricate reasoning across semantic content, layout structures, and visual elements—a task where existing OCR-free approaches struggle to balance model capacity and reasoning accuracy. The authors propose an OCR-free agent framework that formulates the problem as a progressive evidence aggregation process, leveraging thumbnail overviews, semantic retrieval-based navigation, and targeted page acquisition to enable grounded reasoning within a structured working memory. A novel active interactive reasoning mechanism is introduced, combining imitation learning with Group Relative Policy Optimization to efficiently select relevant evidence and generate answers without increasing the number of input pages. Experiments demonstrate that the method outperforms open-source baselines across five benchmarks, approaching the performance of closed-source models and achieving up to a 47.9% improvement over RAG baselines in out-of-domain settings, thereby validating the efficacy of the proposed selective attention mechanism.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Natural Language Processing: Question Answering
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Multi-page Document Visual Question Answering requires reasoning over semantics, layouts, and visual elements in long, visually dense documents. Existing OCR-free methods face a trade-off between capacity and precision: end-to-end models scale poorly with document length, while visual retrieval-based pipelines are brittle and passive. We propose Doc-$V^*$, an \textbf{OCR-free agentic} framework that casts multi-page DocVQA as sequential evidence aggregation. Doc-$V^*$ begins with a thumbnail overview, then actively navigates via semantic retrieval and targeted page fetching, and aggregates evidence in a structured working memory for grounded reasoning. Trained by imitation learning from expert trajectories and further optimized with Group Relative Policy Optimization, Doc-$V^*$ balances answer accuracy with evidence-seeking efficiency. Across five benchmarks, Doc-$V^*$ outperforms open-source baselines and approaches proprietary models, improving out-of-domain performance by up to \textbf{47.9\%} over RAG baseline. Other results reveal effective evidence aggregation with selective attention, not increased input pages.

Problem

• Multi-page Document VQA
• Visual Reasoning
• OCR-free
• Evidence Aggregation
• Long Document Understanding

Innovation

• OCR-free
• agentic reasoning
• evidence aggregation
• visual retrieval
• multi-page DocVQA