MedHal: An Evaluation Dataset for Medical Hallucination Detection

2025-04-11

AI Summary

Current medical hallucination detection lacks large-scale, multi-task evaluation benchmarks, limiting rigorous assessment of model reliability. To address this, we introduce MedHalluBench—the first large-scale, multi-source, multi-task benchmark for medical hallucination detection, built upon real-world clinical texts including electronic health records, biomedical literature, and question-answer pairs. It features fine-grained human annotations and fact-inconsistency attribution explanations grounded in clinical domain knowledge. Methodologically, we integrate collaborative annotation by clinical experts with an interpretable modeling framework that decouples supervision-based training from detection mechanisms. MedHalluBench is the first benchmark to unify scalability, task diversity, and attribution interpretability in medical hallucination detection, filling a critical gap in specialized evaluation resources. Experiments demonstrate that detectors trained on MedHalluBench achieve significantly higher accuracy than general-purpose baselines and reduce reliance on manual review by over 90%, thereby advancing the development of safe and trustworthy medical AI systems.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We present MedHal, a novel large-scale dataset specifically designed to evaluate if models can detect hallucinations in medical texts. Current hallucination detection methods face significant limitations when applied to specialized domains like medicine, where they can have disastrous consequences. Existing medical datasets are either too small, containing only a few hundred samples, or focus on a single task like Question Answering or Natural Language Inference. MedHal addresses these gaps by: (1) incorporating diverse medical text sources and tasks; (2) providing a substantial volume of annotated samples suitable for training medical hallucination detection models; and (3) including explanations for factual inconsistencies to guide model learning. We demonstrate MedHal's utility by training and evaluating a baseline medical hallucination detection model, showing improvements over general-purpose hallucination detection approaches. This resource enables more efficient evaluation of medical text generation systems while reducing reliance on costly expert review, potentially accelerating the development of medical AI research.

Problem

• Detecting hallucinations in medical texts accurately
• Overcoming limitations of small medical datasets
• Providing annotated samples for model training

Innovation

• Diverse medical text sources and tasks
• Large volume of annotated samples
• Explanations for factual inconsistencies