Mirage The Illusion of Visual Understanding

2026-03-23

AI Summary

This study reveals that state-of-the-art multimodal vision-language models exhibit “phantom reasoning” in high-stakes domains such as healthcare—relying heavily on textual priors rather than actual visual input for decision-making. Through controlled experiments and instruction manipulation, the authors demonstrate that leading models achieve top performance on chest X-ray question-answering benchmarks even without image inputs; however, their accuracy drops significantly when explicitly instructed to guess, exposing a spurious dependence on textual cues. The work formally defines and names this phenomenon “phantom reasoning,” highlights critical text-leakage vulnerabilities in current evaluation benchmarks, and introduces B-Clean, a novel framework designed to establish a more rigorous and visually grounded assessment protocol for genuine multimodal understanding.

Technology Category

• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Multimodal AI systems have achieved remarkable performance across a broad range of real-world tasks, yet the mechanisms underlying visual-language reasoning remain surprisingly poorly understood. We report three findings that challenge prevailing assumptions about how these systems process and integrate visual information. First, Frontier models readily generate detailed image descriptions and elaborate reasoning traces, including pathology-biased clinical findings, for images never provided; we term this phenomenon mirage reasoning. Second, without any image input, models also attain strikingly high scores across general and medical multimodal benchmarks, bringing into question their utility and design. In the most extreme case, our model achieved the top rank on a standard chest X-ray question-answering benchmark without access to any images. Third, when models were explicitly instructed to guess answers without image access, rather than being implicitly prompted to assume images were present, performance declined markedly. Explicit guessing appears to engage a more conservative response regime, in contrast to the mirage regime in which models behave as though images have been provided. These findings expose fundamental vulnerabilities in how visual-language models reason and are evaluated, pointing to an urgent need for private benchmarks that eliminate textual cues enabling non-visual inference, particularly in medical contexts where miscalibrated AI carries the greatest consequence. We introduce B-Clean as a principled solution for fair, vision-grounded evaluation of multimodal AI systems.

Problem

• mirage reasoning
• visual-language models
• multimodal evaluation
• non-visual inference
• medical AI

Innovation

• mirage reasoning
• vision-language models
• multimodal evaluation
• B-Clean
• non-visual inference