MedHal-Loc: Are "Explainable-by-Architecture" Medical Hallucination Detectors Faithful Localizers? A Localization Benchmark

2026-06-19

AI Summary

Although existing medical hallucination detectors claim interpretability, their ability to accurately localize erroneous text spans lacks systematic evaluation. This work proposes MedHal-Loc, the first benchmark specifically designed to assess the faithfulness of hallucination localization in medical texts. By combining both synthetically injected and naturally generated data—augmented with controlled experiments and human-annotated gold-standard error spans—we systematically evaluate a range of approaches, including natural language inference (NLI), consistency-based methods, dedicated span detectors, and knowledge graph (KG) triple pipelines. Our results challenge the assumption that detection performance inherently implies reliable localization: while methods such as FAVA and NLI-per-clause significantly outperform random baselines in localization, the KG pipeline, despite achieving a detection F1 of 0.609, shows no significant localization capability due to limited entity coverage (59%).

Abstract

Detecting hallucinations in clinical text is increasingly framed as an explainability problem: systems should not merely flag an unreliable response but point to the offending span. Architectures built around knowledge-graph (KG) triple decomposition are marketed for exactly this auditability, yet their localization ability is typically assumed rather than measured. We introduce MedHal-Loc, a benchmark and metric for localization faithfulness -- whether a detector's top-ranked error unit actually overlaps the erroneous span. The controlled subset comprises 300 PubMedQA-derived statements with single, span-level errors injected across four localizable types (entity substitution, relation error, mechanism misattribution, invention), yielding gold spans by construction; a complementary natural subset documents that real hallucinations are dominated by diffuse conclusion-flips that resist span localization (a human expert accepted 1/18 candidate spans). Evaluating four fine-grained paradigms, we find that NLI-per-clause, consistency-per-sentence, and the dedicated span detector FAVA all localize well above chance, whereas an elaborate KG-triple pipeline localizes no better than chance (+3.3pp, n.s.), bottlenecked by ~59% entity-extraction coverage -- despite competitive detection F1 (0.609). Detection competence does not imply faithful localization; architectural explainability must be validated, not presumed.

Problem

• medical hallucination
• localization
• faithfulness
• explainability
• benchmark

Innovation

• localization faithfulness
• medical hallucination detection
• explainable-by-architecture
• knowledge-graph triples
• MedHal-Loc benchmark