Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment

2026-07-16

AI Summary

This work addresses the inefficiency of existing multimodal document question answering methods, which often rely on extensive annotated data or redundant reasoning steps. The authors propose Perception-RFT, a training framework that leverages Group Relative Policy Optimization (GRPO) to directly align visual features with structured localization outputs, thereby eliminating intermediate reasoning stages. Notably, they observe for the first time in 4B-scale models an emergent convergence toward inference-free strategies. They further demonstrate that transitioning early from supervised fine-tuning (SFT) to reinforcement learning substantially reduces data requirements. The proposed approach achieves over 60% reduction in reasoning token length while maintaining both semantic robustness and geometric accuracy on out-of-distribution (OOD) benchmarks, matching prior localization performance with 65% less training data.

Abstract

Efficient multimodal document question answering with explicit visual grounding, locating the precise document region that supports each answer remains an open challenge. Current approaches bifurcate into Supervised Fine-Tuning (SFT), which requires large annotated datasets and reaches optimization plateaus, and reasoning-centric Reinforcement Learning (RL), which depends on verbose intermediate traces that inflate inference token cost without clear benefit. We introduce Perception-RFT, a training framework that applies Group Relative Policy Optimization (GRPO) to multimodal document QA, bypassing intermediate reasoning tokens to directly align visual features with structured grounding outputs. To rigorously evaluate the necessity of reasoning, we construct a reasoning variant under identical reward settings. We find that reasoning-enabled models suppress their reasoning traces during training, converging to direct perception-based policies at the 4B parameter scale, reducing per-query inference token length by more than 60%, while reasoning-enabled RL underperforms perception-only training. Through a fine-grained analysis of Qwen3-VL-4B optimization dynamics, we confirm that SFT saturation and cold-start RL instability established in text-domain post-training extend to multimodal, and identify a previously uncharacterized Grounding Divergence: a selective trade-off between semantic robustness and geometric precision on two out of distribution (OOD) benchmarks (4,828 samples) under joint RL optimization. We further show that an early SFT$\rightarrow$RL transition achieves comparable precision with 65% less training data.

Problem

• multimodal document question answering
• visual grounding
• post-training
• reasoning-free alignment
• reinforcement learning

Innovation

• Perception-RFT
• reasoning-free alignment
• multimodal document QA
• visual grounding
• Group Relative Policy Optimization