Hallucination Detection and Correction in Medical VLMs via Counter-Evidence Verification

2026-06-16

AI Summary

Medical vision-language models (VLMs) often generate hallucinated content unsupported by visual evidence, undermining diagnostic reliability, yet existing approaches struggle to effectively verify consistency between generated text and underlying image data. This work proposes CoEV, a training-free, plug-and-play framework that introduces, for the first time, a four-quadrant diagnostic diagram to detect and post-process hallucinations through counterfactual verification, attention-region alignment, and bidirectional consistency checks. Evaluated across four medical imaging datasets, CoEV substantially outperforms current methods, achieving relative improvements of 3.0% in PR-AUC and 3.9% in ROC-AUC. Furthermore, its hallucination correction capability yields up to a 12.5% increase in Micro-F1 and reduces report-level hallucination rates by over 11.9%.

Abstract

Vision-Language models (VLMs) reliability in medical diagnosis is challenged by trust-undermining hallucinations. Existing hallucination detection approaches mainly focus on identifying factual inconsistencies between generated text and reference data. While some studies analyze where models attend in images, they seldom verify whether such attention truly reflects the visual evidence supporting the generated text. To address this gap, we propose Co}unter-Evidence Verification (CoEV), a training-free plug-and-play framework that detects and corrects hallucinations through evidence-based factual consistency verification. CoEV performs bidirectional verification between textual assertions and visual evidence, testing whether each statement is supported by its corresponding evidence region, and assigns each statement into a four-quadrant diagnostic map capturing combinations of text factuality and visual grounding. CoEV detects hallucinated content and serves as a post hoc refinement tool, correcting hallucinations without retraining. Extensive experiments on four medical datasets show that CoEV combats hallucinations in VLMs.For hallucination detection, CoEV consistently outperforms existing methods, improving average PR-AUC and ROC-AUC by 3.0% and 3.9% absolute points respectively, with notable gains of up to 18.5% in specific VQA scenarios. For hallucination correction, it improves Micro-F1 by up to 12.5%, reduces hallucination rates by over 11.9% on medical report generation, and also boosts medical VQA accuracy. These results show that CoEV enables reliable detection and correction of hallucinations, providing clinicians with dependable, evidence-based cues for diagnosis. Code will be released upon acceptance.

Problem

• hallucination
• medical VLMs
• factual consistency
• visual grounding
• evidence verification

Innovation

• Counter-Evidence Verification
• Hallucination Detection
• Vision-Language Models
• Medical AI
• Factual Consistency