V-tableR1: Process-Supervised Multimodal Table Reasoning with Critic-Guided Policy Optimization

2026-04-22

AI Summary

This work addresses the tendency of multimodal large language models to rely on superficial pattern matching rather than verifiable multi-step logical reasoning, particularly in abstract visual reasoning tasks where pixel-level alignment introduces ambiguity. To tackle this, the authors propose V-tableR1, a framework that leverages the structured grid of tables as an ideal setting for rigorous reasoning. V-tableR1 employs a discriminative vision-language model to provide stepwise feedback on explicit visual chains of thought generated by a policy model and, for the first time, effectively applies process-supervised reinforcement learning to multimodal table reasoning. The proposed PGPO algorithm integrates process rewards, decoupled policy constraints, and length-aware dynamic sampling, substantially enhancing logical verifiability. Experiments show that V-tableR1-4B achieves state-of-the-art performance among open-source models on complex table reasoning benchmarks, outperforming its supervised fine-tuning baseline and surpassing models with 18× more parameters, while effectively mitigating visual hallucinations and shortcut guessing.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We introduce V-tableR1, a process-supervised reinforcement learning framework that elicits rigorous, verifiable reasoning from multimodal large language models (MLLMs). Current MLLMs trained solely on final outcomes often treat visual reasoning as a black box, relying on superficial pattern matching rather than performing rigorous multi-step inference. While Reinforcement Learning with Verifiable Rewards could enforce transparent reasoning trajectories, extending it to visual domains remains severely hindered by the ambiguity of grounding abstract logic into continuous pixel space. We solve this by leveraging the deterministic grid structure of tables as an ideal visual testbed. V-tableR1 employs a specialized critic VLM to provide dense, step-level feedback on the explicit visual chain-of-thought generated by a policy VLM. To optimize this system, we propose Process-Guided Direct Alignment Policy Optimization (PGPO), a novel RL algorithm integrating process rewards, decoupled policy constraints, and length-aware dynamic sampling. Extensive evaluations demonstrate that V-tableR1 explicitly penalizes visual hallucinations and shortcut guessing. By fundamentally shifting multimodal inference from black-box pattern matching to verifiable logical derivation, V-tableR1 4B establishes state-of-the-art accuracy among open-source models on complex tabular benchmarks, outperforming models up to 18x its size and improving over its SFT baseline

Problem

• multimodal reasoning
• visual hallucination
• table understanding
• process supervision
• reinforcement learning

Innovation

• process-supervised reinforcement learning
• multimodal table reasoning
• critic-guided policy optimization
• visual chain-of-thought
• verifiable reasoning