What, Whether and How? Unveiling Process Reward Models for Thinking with Images Reasoning

2026-02-09

AI Summary

This work addresses the absence of fine-grained evaluation benchmarks for process reward models (PRMs) tailored to the “reasoning with visual grounding” paradigm, which hinders effective identification of diverse errors in visual reasoning. To bridge this gap, we construct the first PRM evaluation benchmark for this paradigm, comprising 1,206 human-annotated high-quality reasoning trajectories and defining seven fine-grained error categories. We systematically evaluate the capability of leading large vision-language models (LVLMs) to serve as PRMs in assessing reasoning processes. Our experiments reveal significant limitations: LVLM-based PRMs exhibit substantial performance variation across error types and are adversely affected by positivity bias and positional sensitivity, undermining their reliability in modeling process-level rewards.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The rapid advancement of Large Vision Language Models (LVLMs) has demonstrated excellent abilities in various visual tasks. Building upon these developments, the thinking with images paradigm has emerged, enabling models to dynamically edit and re-encode visual information at each reasoning step, mirroring human visual processing. However, this paradigm introduces significant challenges as diverse errors may occur during reasoning processes. This necessitates Process Reward Models (PRMs) for distinguishing positive and negative reasoning steps, yet existing benchmarks for PRMs are predominantly text-centric and lack comprehensive assessment under this paradigm. To address these gaps, this work introduces the first comprehensive benchmark specifically designed for evaluating PRMs under the thinking with images paradigm. Our main contributions are: (1) Through extensive analysis of reasoning trajectories and guided search experiments with PRMs, we define 7 fine-grained error types and demonstrate both the necessity for specialized PRMs and the potential for improvement. (2) We construct a comprehensive benchmark comprising 1,206 manually annotated thinking with images reasoning trajectories spanning 4 categories and 16 subcategories for fine-grained evaluation of PRMs. (3) Our experimental analysis reveals that current LVLMs fall short as effective PRMs, exhibiting limited capabilities in visual reasoning process evaluation with significant performance disparities across error types, positive evaluation bias, and sensitivity to reasoning step positions. These findings demonstrate the effectiveness of our benchmark and establish crucial foundations for advancing PRMs in LVLMs.

Problem

• Process Reward Models
• Thinking with Images
• Visual Reasoning
• Error Types
• Large Vision Language Models

Innovation

• Process Reward Models
• Thinking with Images
• Large Vision Language Models
• Reasoning Trajectories
• Visual Reasoning Evaluation