PARALLAX: Separating Genuine Hallucination Detection from Benchmark Construction Artifacts

2026-05-16

AI Summary

This study addresses the potential overestimation of hallucination detection performance due to dataset construction artifacts—particularly prompt leakage—in existing benchmarks. Through a systematic evaluation of 22 detection methods, 12 open-source models, and 6 corpora, the work quantifies, for the first time, the extent to which such artifacts inflate reported results. To enable reliable real-time hallucination detection, the authors propose DRIFT, a supervised probing method based on transitions in upper-layer hidden states. Experimental findings reveal that, once prompt leakage is controlled, most existing approaches perform near random chance, with only SAPLMA and DRIFT demonstrating consistent effectiveness across diverse settings. These results indicate that current progress in hallucination detection has been substantially overstated and establish a more trustworthy evaluation framework for real-world applications.

Abstract

Large language models (LLMs) hallucinate with confidence: their outputs can be fluent, authoritative, and simply wrong. In medical, legal, and scientific applications this failure causes direct harm, and detecting it from internal model states offers a path to safer deployment. A growing body of work reports that this problem is increasingly tractable, with recent methods achieving high detection performance on widely used benchmarks. We show, however, that much of this apparent progress does not survive scrutiny. Four of the six corpora embed the ground-truth answer directly in the input prompt. A naïve text-similarity baseline we call \textsc{TxTemb} exploits this to achieve near-perfect detection scores without any access to model internals. To measure what genuine detection capability remains once these artifacts are controlled, we conduct a large-scale evaluation spanning twenty-two detection methods, twelve open-source models spanning six architectural families, and six corpora. We further introduce \textbf{DRIFT}, a supervised probe over inter-layer hidden-state transitions, as a point of comparison for live-generation detection. Our findings suggest that the field's reported progress on hallucination detection is substantially explained by benchmark construction artifacts in widely used corpora, and that the majority of established baselines perform near chance under controlled conditions; the consistent exceptions are SAPLMA and DRIFT, both supervised probes on upper-layer hidden states.

Problem

• hallucination detection
• benchmark artifacts
• large language models
• model internals
• evaluation bias

Innovation

• hallucination detection
• benchmark artifacts
• hidden-state probing
• DRIFT
• large language models