VIHD: Visual Intervention-based Hallucination Detection for Medical Visual Question Answering

2026-05-20

AI Summary

This work addresses the critical issue of hallucinations in medical multimodal large language models (MLLMs) during visual question answering (VQA), where responses often lack grounding in visual evidence, thereby jeopardizing clinical decision safety. To mitigate this, the authors propose a fine-grained visual dependency modeling approach for hallucination detection and suppression. Their method employs a Visual Dependency Probe (VDP) to identify key decoding layers, leverages Calibrated Semantic Entropy (CSE) to accurately detect hallucinatory outputs, and applies Visual token-masking Intervention during Decoding (VID) to suppress them. Unlike conventional heuristic input-perturbation strategies, this framework operates directly within the decoding process. Extensive experiments on three medical VQA benchmarks and two prominent medical MLLMs demonstrate substantial improvements over state-of-the-art methods, confirming both its effectiveness and generalizability.

Abstract

While medical Multimodal Large Language Models (MLLMs) have shown promise in assisting diagnosis, they still frequently generate hallucinated responses that appear linguistically plausible but lack visual evidence. Such hallucinations pose risks to clinical decision-making and necessitate effective detection. Existing introspective detection methods primarily perform uncertainty estimation or logical verification by analyzing model responses conditioned on original or perturbed inputs. However, such external perturbations are often heuristic and context-agnostic, which overlooks the internal cross-modal dependency between generated tokens and related visual tokens during decoding. To address this issue, we propose VIHD, a Visual Intervention-based Hallucination Detection method that leverages targeted visual token masking to calibrate semantic entropy for more effective hallucination detection. VIHD locates visually dominant decoder layers via Visual Dependency Probing (VDP), executes Visual Intervention Decoding (VID) via token masking to calibrate the semantic distribution, and quantifies the resulting Calibrated Semantic Entropy (CSE) as a reliable hallucination signal. Extensive experiments on three medical VQA benchmarks with two medical MLLMs demonstrate that VIHD consistently outperforms state-of-the-art methods, underscoring the importance of fine-grained visual dependency for hallucination detection. The code will be available at https://github.com/Jiayi-Chen-AU/VIHD

Problem

• hallucination detection
• medical visual question answering
• multimodal large language models
• visual evidence
• clinical decision-making

Innovation

• Visual Intervention
• Hallucination Detection
• Semantic Entropy Calibration
• Medical VQA
• Cross-modal Dependency