Visuals Lie, Consistency Speaks: Disentangling Spatial Attention from Reliability in Vision-Language Models

2026-06-15

AI Summary

This study challenges the prevailing assumption that spatial attention in vision-language models (VLMs) reflects response reliability. Through a systematic analysis employing structured attention metrics—such as cluster count $C_k$, spatial entropy $H_s$, and its inter-layer variation $\Delta H_s$—combined with self-consistency evaluation, causal interventions, and hidden-state probing across multiple models, the work reveals that spatial attention exhibits virtually no correlation with accuracy ($R \approx 0.001$). In contrast, self-consistency emerges as a strong predictor of truthfulness ($R = 0.429$). Furthermore, the investigation uncovers fundamental architectural differences in reliability mechanisms: LLaVA relies on a fragile late-stage bottleneck, whereas PaliGemma and Qwen2-VL demonstrate global robustness, highlighting divergent pathways to trustworthy reasoning in VLMs.

Abstract

Multimodal Foundation Models are increasingly used as reasoning agents, making reliability, knowing when a model may hallucinate, critical. A common intuition, which we call the Attention-Confidence Assumption, holds that reliability follows from "structural" visual perception: tight attention on relevant regions should signal a trustworthy answer, while scattered attention signals confusion. We challenge this through the VLM Reliability Probe (VRP), a systematic cross-family study of reliability signals in contemporary Vision-Language Models (VLMs). We introduce structural-attention metrics, cluster counts (C_k) and spatial entropy (H_s), to quantify the visual encoder's gaze, and track its evolution (Delta H_s) across layers. This reveals a "Symbolic Detachment": models often "Early Lock" visual features only to diffuse attention later, severing early perception from final generation. Contrary to the grounding hypothesis, we find a "Cluster Failure": spatial attention has near-zero correlation (R approx 0.001) with accuracy. Instead, reliability is a phenomenon of generation dynamics and internal-state distributions. Self-Consistency, the agreement rate across sampled reasoning paths, is the dominant predictor of truth (R = 0.429). Scaling causal interventions exposes a sharp architectural divergence: LLaVA locks its prediction in a fragile late-stage bottleneck, whereas PaliGemma and Qwen2-VL distribute reliability globally, staying resilient even when ~50% or more of their most predictive layer is destroyed. For current VLMs, reliability signals are detached from visual grounding maps and are best inferred from generation-time dynamics and hidden-state probes.

Problem

• Vision-Language Models
• Reliability
• Spatial Attention
• Hallucination
• Self-Consistency

Innovation

• Self-Consistency
• Spatial Attention
• Reliability Probing
• Symbolic Detachment
• Vision-Language Models