Semantic Uncertainty Quantification of Hallucinations in LLMs: A Quantum Tensor Network Based Method

2026-01-27

AI Summary

This work proposes a novel framework for quantifying semantic uncertainty in large language models (LLMs) to mitigate hallucination—where outputs appear fluent yet semantically inconsistent. For the first time, quantum tensor networks are leveraged to construct a semantic uncertainty quantification framework that analyzes the probability distribution of generated sequences through semantic equivalence clustering. By integrating an entropy maximization strategy, the method identifies high-confidence, semantically coherent outputs while flagging high-risk regions requiring human intervention. The approach balances interpretability and practicality, demonstrating robust performance even under resource constraints. Extensive evaluation across four datasets—including TriviaQA and Natural Questions—encompassing 116 experiments on eight mainstream LLM architectures shows significant improvements over existing baselines, with superior AUROC and AURAC metrics.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: (Large) Language Models
• Reasoning under Uncertainty: Uncertainty Representations

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large language models (LLMs) exhibit strong generative capabilities but remain vulnerable to confabulations, fluent yet unreliable outputs that vary arbitrarily even under identical prompts. Leveraging a quantum tensor network based pipeline, we propose a quantum physics inspired uncertainty quantification framework that accounts for aleatoric uncertainty in token sequence probability for semantic equivalence based clustering of LLM generations. This offers a principled and interpretable scheme for hallucination detection. We further introduce an entropy maximization strategy that prioritizes high certainty, semantically coherent outputs and highlights entropy regions where LLM decisions are likely to be unreliable, offering practical guidelines for when human oversight is warranted. We evaluate the robustness of our scheme under different generation lengths and quantization levels, dimensions overlooked in prior studies, demonstrating that our approach remains reliable even in resource constrained deployments. A total of 116 experiments on TriviaQA, NQ, SVAMP, and SQuAD across multiple architectures including Mistral-7B, Mistral-7B-instruct, Falcon-rw-1b, LLaMA-3.2-1b, LLaMA-2-13b-chat, LLaMA-2-7b-chat, LLaMA-2-13b, and LLaMA-2-7b show consistent improvements in AUROC and AURAC over state of the art baselines.

Problem

• hallucination
• semantic uncertainty
• large language models
• confabulation
• uncertainty quantification

Innovation

• Quantum Tensor Network
• Semantic Uncertainty Quantification
• Hallucination Detection
• Aleatoric Uncertainty
• Entropy Maximization