HalluTracer: Hallucination Detection via Depth-Averaging Truth Signals

2026-08-17

AI Summary

This study addresses the limitation of existing white-box hallucination detection methods that overlook cross-layer truthfulness signals in large language models. We propose a Depth Averaging framework that exploits the geometric sparsity of layer-wise signals to reconstruct the detection paradigm from single-layer selection to depth aggregation. This approach effectively extracts linearly separable truthfulness features while suppressing noise. Extensive evaluations across six open-source models and five benchmarks demonstrate that our framework significantly outperforms current white-box baselines, achieving performance gains of 1 to 14 percentage points. These results validate the critical role of cross-layer signal aggregation in hallucination detection and offer a novel perspective for advancing white-box interpretability and reliability assessment in large language models.

Abstract

Even well-aligned large language models confidently generate factually incorrect text, making hallucination a persistent reliability risk in high-stakes deployments. These models nonetheless carry linearly separable truthfulness signals in their internal representations. Existing white-box detectors, however, collapse this evidence to isolated components or a single depth, discarding discriminative information distributed across the full forward pass. We introduce HalluTracer, a detection framework that reads and aggregates truthfulness evidence across every layer of the forward pass before the model emits any answer token. A geometric analysis reveals that the per-layer signals are weakly correlated, so that simple depth averaging suppresses layer-specific noise and captures nearly all linearly accessible information. Across six open-source language models and five hallucination benchmarks, HalluTracer consistently outperforms matched white-box baselines, with gains ranging from one to fourteen points. Collectively, our work recasts hallucination detection from a layer-selection problem into a depth-aggregation problem governed by the geometric sparsity of the truthfulness signal.

Problem

• Hallucination Detection
• Large Language Models
• Truthfulness Signals
• White-box Detection
• Internal Representations

Innovation

• Hallucination Detection
• Depth-Averaging
• Truth Signals
• Geometric Sparsity
• White-box Detector