A Benchmark for Hallucination Detection in VLMs for Gastrointestinal Endoscopy

2026-06-23

AI Summary

This study addresses the susceptibility of current vision-language models (VLMs) to hallucination in gastrointestinal endoscopy applications, where no targeted evaluation benchmark previously existed. The authors introduce Gut-VLM, the first hallucination detection benchmark specifically designed for gastrointestinal endoscopic visual question answering (VQA). They systematically evaluate nine detection methods—categorized as black-box, gray-box, and white-box—across five medical VLMs. Experimental results demonstrate that white-box approaches significantly outperform other paradigms, with ReXTrust achieving the best performance across all models, reaching a peak AUC of 93.0 and surpassing non-white-box methods by an average of 19.5 AUC points, while still maintaining a robust AUC of 79.9 on LLaVA-v1.6-7B. The study further identifies “confident fabrication” as a systematic failure mode of VLMs in this clinical context.

Abstract

Vision-language models (VLMs) are prone to hallucination, which remains a major barrier to their safe deployment in clinical practice. To date, most hallucination detection methods have been evaluated on radiology benchmarks such as MIMIC-CXR and VQA-RAD, while gastrointestinal (GI) endoscopy remains largely underexplored. In this paper, we benchmark nine hallucination detection methods on the Gut-VLM dataset, a GI diagnostic Visual Question Answering (VQA) dataset with 4,392 test VQA pairs, across five VLMs (MedGemma-4B, MedGemma-27B, LLaVA-Med-7B, LLaVA-v1.6-7B, and Lingshu-32B). The methods span three categories: black-box methods (RadFlag, SelfCheckGPT-NLI), gray-box methods (AvgProb, AvgEnt, MaxProb, MaxEnt, Semantic Entropy, and VASE), and a white-box method (ReXTrust). Our results show that ReXTrust, a white-box method, achieves the highest AUC across all five models, outperforming the strongest alternative method on each VLM by a statistically significant margin (paired permutation test, p < 0.001 in all cases), reaching a peak AUC of 93.0 on MedGemma-4B. White-box hidden-state access provides a consistent advantage of 19.5 AUC points on average (range: 9.5--33.5), with ReXTrust maintaining strong performance even on LLaVA-v1.6-7B (AUC 79.9), where black-box methods and clustering-based gray-box methods collapse to near-chance performance. Among non-white-box methods, token-level gray-box statistics (MaxEnt, MaxProb) are the strongest alternatives, outperforming both clustering-based gray-box methods (Semantic Entropy, VASE) and black-box approaches on average. We further identify confident confabulation, a failure mode in which models hallucinate with high inter-sample consistency or high token-level probability, as a systemic failure for both consistency and uncertainty-based methods.

Problem

• hallucination detection
• vision-language models
• gastrointestinal endoscopy
• visual question answering
• clinical safety

Innovation

• hallucination detection
• vision-language models
• gastrointestinal endoscopy
• white-box methods
• confident confabulation