ClinHallu: A Benchmark for Diagnosing Stage-Wise Hallucinations in Medical MLLM Reasoning

2026-06-12

AI Summary

This work addresses the lack of fine-grained localization of hallucination sources in current medical multimodal large language models (MLLMs). To this end, the authors introduce ClinHallu, a novel benchmark that enables stage-level provenance diagnosis of hallucinations by structuring the reasoning process into three distinct phases: visual perception, knowledge retrieval, and reasoning integration. The framework incorporates stage-replacement interventions and trajectory-supervised fine-tuning to specifically mitigate hallucinations originating from targeted stages. Evaluation on a publicly released benchmark comprising 7,031 expert-annotated samples demonstrates that the proposed approach significantly reduces stage-specific hallucination rates, thereby providing effective support for fine-grained diagnosis and optimization of medical MLLMs.

Abstract

Building trustworthy medical multimodal large language models (MLLMs) is critical for reliable clinical decision support. Existing medical hallucination benchmarks mainly focus on data collection, but often ignore where hallucinations originate within the reasoning process. We find that hallucination sources vary across samples: errors may arise from visual misrecognition, incorrect medical knowledge recall, or flawed reasoning integration. To enable source-level hallucination diagnosis, we introduce ClinHallu, a benchmark for stage-wise hallucination diagnosis in medical MLLM reasoning. ClinHallu contains 7,031 validated instances, where each instance is augmented with a structured reasoning trace decomposed into Visual Recognition, Knowledge Recall, and Reasoning Integration. We also use stage-replacement interventions to measure how correcting specific stages affects the final answer. Beyond evaluation, we show that trace-supervised fine-tuning reduces stage-wise hallucinations. ClinHallu provides a fine-grained hallucination testbed for diagnosing and mitigating reasoning failures in medical MLLMs. The benchmark is publicly available at https://github.com/alibaba-damo-academy/ClinHallu.

Problem

• medical MLLMs
• hallucination diagnosis
• reasoning process
• stage-wise hallucinations
• clinical decision support

Innovation

• stage-wise hallucination diagnosis
• medical MLLM
• structured reasoning trace
• hallucination mitigation
• benchmark