Fine-Grained Uncertainty Decomposition in Large Language Models: A Spectral Approach

2025-09-26

AI Summary

Addressing the challenge of reliably disentangling aleatoric from epistemic uncertainty in large language models (LLMs), this paper proposes a spectral-analysis-based fine-grained uncertainty decomposition method. It introduces, for the first time, von Neumann entropy—originally from quantum information theory—into LLM uncertainty modeling, establishing a semantic similarity-aware frequency-domain decomposition framework. Specifically, spectral entropy is employed to characterize the model’s output distribution in the frequency domain, while joint integration with semantic embeddings enables interpretable separation of the two uncertainty types. Evaluated on multiple state-of-the-art LLMs (e.g., Llama, Qwen) and benchmark datasets (UncertaintyQA, AmbiQA), the method achieves superior performance over existing approaches in both total and aleatoric uncertainty estimation, attaining new state-of-the-art results. This work establishes a novel paradigm for trustworthy LLM inference grounded in principled uncertainty quantification.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

As Large Language Models (LLMs) are increasingly integrated in diverse applications, obtaining reliable measures of their predictive uncertainty has become critically important. A precise distinction between aleatoric uncertainty, arising from inherent ambiguities within input data, and epistemic uncertainty, originating exclusively from model limitations, is essential to effectively address each uncertainty source. In this paper, we introduce Spectral Uncertainty, a novel approach to quantifying and decomposing uncertainties in LLMs. Leveraging the Von Neumann entropy from quantum information theory, Spectral Uncertainty provides a rigorous theoretical foundation for separating total uncertainty into distinct aleatoric and epistemic components. Unlike existing baseline methods, our approach incorporates a fine-grained representation of semantic similarity, enabling nuanced differentiation among various semantic interpretations in model responses. Empirical evaluations demonstrate that Spectral Uncertainty outperforms state-of-the-art methods in estimating both aleatoric and total uncertainty across diverse models and benchmark datasets.

Problem

• Decomposing predictive uncertainty into aleatoric and epistemic components
• Quantifying uncertainty using spectral methods from quantum information theory
• Providing fine-grained semantic differentiation in language model uncertainty

Innovation

• Spectral Uncertainty uses Von Neumann entropy
• Decomposes total uncertainty into distinct components
• Incorporates fine-grained semantic similarity representation