Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?

2026-05-21

AI Summary

Current benchmarks for vision-language models inadequately capture the extent to which these models genuinely rely on fine-grained visual information. This study investigates model sensitivity to partial loss of visual evidence through systematic ablation experiments—such as image token removal and localized occlusion—and multi-level representational analyses, including decision-level behavior and the geometric structure of visual tokens. The findings reveal that models are generally insensitive to the loss of fine-grained visual details, with deep visual representations exhibiting significant homogenization. This suggests that prevailing accuracy metrics substantially overestimate models’ visual grounding capabilities. The work thus exposes critical limitations in current evaluation paradigms and offers a novel perspective and methodological foundation for assessing fine-grained visual understanding.

Abstract

Benchmark accuracy is often implicitly assumed to reflect grounded visual understanding in vision-language models (VLMs), yet it remains unclear to what extent such scores truly reflect reliance on visual evidence. Motivated by a surprising observation that removing a substantial fraction of image tokens only degrades model performance very slightly on a widely used hallucination benchmark, we systematically investigate this mismatch in a set of open-source VLMs. Our analysis spans multiple levels of granularity, spanning global visual degradation, localized occlusion, question reformulation, answer-space expansion, and decision-level analyses beyond standard accuracy. We further complement these behavioral results with a layer-wise analysis of vision-token geometry. Throughout the experiments, we find that although VLMs do incorporate visual input, their predictions are less sensitive to the loss of fine-grained visual evidence that standard accuracy should have suggested. Even when the final prediction remains unchanged, the model's internal support for the correct answer may already be weakened. We further complement a representation-level analysis, which shows increasing similarity among visual tokens in deeper layers, providing a possible explanation for our findings. Together, these results suggest that current benchmarks are not sufficient to reliably evaluate fine-grained visual grounding in VLMs.

Problem

• vision-language models
• visual grounding
• benchmark evaluation
• hallucination
• visual understanding

Innovation

• vision-language models
• visual grounding
• benchmark evaluation
• token ablation
• representation geometry