Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models

2026-04-11

AI Summary

This work addresses the susceptibility of multimodal large language models to hallucination in long-chain reasoning, which often stems from failed visual semantic anchoring and overreliance on linguistic priors rather than visual evidence. The authors propose V-STAR, a novel training paradigm that identifies and mitigates the previously unexamined “Reasoning–Vision Truth Disconnection” (RVTD) phenomenon. By extending supervision signals from the output layer into internal attention mechanisms, V-STAR leverages a GRPO framework integrating Hierarchical Visual Attention Rewards (HVAR) and a Forced Reflection Mechanism (FRM). This approach dynamically guides critical intermediate layers to refocus on visual inputs at high-entropy cognitive branching points, enabling lightweight, end-to-end enhancement of visual grounding. Experiments demonstrate that V-STAR substantially reduces hallucination rates and improves accuracy and reliability on complex visual reasoning tasks.

Technology Category

• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Multimodal Large Reasoning Models (MLRMs) have achieved remarkable strides in visual reasoning through test time compute scaling, yet long chain reasoning remains prone to hallucinations. We identify a concerning phenomenon termed the Reasoning Vision Truth Disconnect (RVTD): hallucinations are strongly correlated with cognitive bifurcation points that often exhibit high entropy states. We attribute this vulnerability to a breakdown in visual semantic anchoring, localized within the network's intermediate layers; specifically, during these high uncertainty transitions, the model fails to query visual evidence, reverting instead to language priors. Consequently, we advocate a shift from solely outcome level supervision to augmenting it with fine grained internal attention guidance. To this end, we propose V-STAR (Visual Structural Training with Attention Reinforcement), a lightweight, holistic training paradigm designed to internalize visually aware reasoning capabilities. Central to our approach is the Hierarchical Visual Attention Reward (HVAR), integrated within the GRPO framework. Upon detecting high entropy states, this mechanism dynamically incentivizes visual attention across critical intermediate layers, thereby anchoring the reasoning process back to the visual input. Furthermore, we introduce the Forced Reflection Mechanism (FRM), a trajectory editing strategy that disrupts cognitive inertia by triggering reflection around high entropy cognitive bifurcation points and encouraging verification of subsequent steps against the visual input, thereby translating external debiasing interventions into an intrinsic capability for hallucination mitigation.

Problem

• hallucination
• multimodal reasoning
• visual anchoring
• cognitive bifurcation
• reasoning vision truth disconnect

Innovation

• multimodal reasoning
• hallucination mitigation
• visual attention reinforcement
• cognitive bifurcation
• internal supervision