Learning Uncertainty from Sequential Internal Dispersion in Large Language Models

2026-04-17

AI Summary

Existing methods for estimating uncertainty based on internal states of large language models often rely on strong assumptions and neglect sequential information, limiting their effectiveness in hallucination detection. This work proposes SIVR, a lightweight, task-agnostic framework that eschews restrictive assumptions about hidden state dynamics and instead leverages variance sequences of per-token representations across model layers. By learning temporal patterns from these variance trajectories, SIVR enables effective hallucination detection without requiring extensive training data. The method demonstrates significant performance gains over strong baselines across multiple benchmarks, exhibiting superior generalization and practical deployment potential.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Uncertainty estimation is a promising approach to detect hallucinations in large language models (LLMs). Recent approaches commonly depend on model internal states to estimate uncertainty. However, they suffer from strict assumptions on how hidden states should evolve across layers, and from information loss by solely focusing on last or mean tokens. To address these issues, we present Sequential Internal Variance Representation (SIVR), a supervised hallucination detection framework that leverages token-wise, layer-wise features derived from hidden states. SIVR adopts a more basic assumption that uncertainty manifests in the degree of dispersion or variance of internal representations across layers, rather than relying on specific assumptions, which makes the method model and task agnostic. It additionally aggregates the full sequence of per-token variance features, learning temporal patterns indicative of factual errors and thereby preventing information loss. Experimental results demonstrate SIVR consistently outperforms strong baselines. Most importantly, SIVR enjoys stronger generalisation and avoids relying on large training sets, highlighting the potential for practical deployment. Our code repository is available online at https://github.com/ponhvoan/internal-variance.

Problem

• uncertainty estimation
• hallucination detection
• large language models
• internal representations
• information loss

Innovation

• uncertainty estimation
• hallucination detection
• internal representation variance
• large language models
• model-agnostic