🤖 AI Summary
Factuality errors in large language model (LLM) question answering—primarily caused by hallucination—are inadequately captured by existing entropy-based semantic uncertainty estimation methods, which suffer from sampling noise and clustering instability induced by variable-length outputs.
Method: We propose Semantic Reconstruction Entropy (SRE), a robust uncertainty quantification framework that first augments input-side semantic diversity via faithful paraphrasing, then applies energy-driven progressive hybrid clustering in semantic space to enable adaptive, length-agnostic grouping and uncertainty estimation.
Contribution/Results: SRE eliminates sensitivity to output length and sampling strategy. Extensive experiments on SQuAD and TriviaQA demonstrate that SRE significantly outperforms state-of-the-art baselines in hallucination detection accuracy, stability under distributional shift, and cross-dataset generalization—achieving consistent improvements across all metrics.
📝 Abstract
Reliable question answering with large language models (LLMs) is challenged by hallucinations, fluent but factually incorrect outputs arising from epistemic uncertainty. Existing entropy-based semantic-level uncertainty estimation methods are limited by sampling noise and unstable clustering of variable-length answers. We propose Semantic Reformulation Entropy (SRE), which improves uncertainty estimation in two ways. First, input-side semantic reformulations produce faithful paraphrases, expand the estimation space, and reduce biases from superficial decoder tendencies. Second, progressive, energy-based hybrid clustering stabilizes semantic grouping. Experiments on SQuAD and TriviaQA show that SRE outperforms strong baselines, providing more robust and generalizable hallucination detection. These results demonstrate that combining input diversification with multi-signal clustering substantially enhances semantic-level uncertainty estimation.