Semantic Faithfulness and Entropy Production Measures to Tame Your LLM Demons and Manage Hallucinations

2025-12-03

AI Summary

Large language models (LLMs) suffer from semantic hallucinations, leading to unfaithful outputs during task execution. Method: We propose the first unsupervised evaluation framework grounded in information theory and non-equilibrium thermodynamics. We model the LLM as a bipartite information engine, invoke a Maxwell’s demon mechanism to characterize hidden-layer regulation of semantic flow, and formulate a semantic transformation dynamics model over “Question–Context–Answer” (QCA) triples. We define two complementary metrics: semantic faithfulness and semantic entropy production. Quantification is achieved via topic probability distribution modeling, transition matrix inference, KL-divergence-based convex optimization, and temporal information entropy analysis. Results: Evaluated on SEC 10-K financial report summarization, the two metrics exhibit significant negative correlation: higher faithfulness corresponds to lower hallucination rates. They support both independent and joint assessment. This work establishes an interpretable, annotation-free paradigm for evaluating LLM trustworthiness.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Evaluation and Analysis

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Evaluating faithfulness of Large Language Models (LLMs) to a given task is a complex challenge. We propose two new unsupervised metrics for faithfulness evaluation using insights from information theory and thermodynamics. Our approach treats an LLM as a bipartite information engine where hidden layers act as a Maxwell demon controlling transformations of context $C $ into answer $A$ via prompt $Q$. We model Question-Context-Answer (QCA) triplets as probability distributions over shared topics. Topic transformations from $C$ to $Q$ and $A$ are modeled as transition matrices ${f Q}$ and ${f A}$ encoding the query goal and actual result, respectively. Our semantic faithfulness (SF) metric quantifies faithfulness for any given QCA triplet by the Kullback-Leibler (KL) divergence between these matrices. Both matrices are inferred simultaneously via convex optimization of this KL divergence, and the final SF metric is obtained by mapping the minimal divergence onto the unit interval [0,1], where higher scores indicate greater faithfulness. Furthermore, we propose a thermodynamics-based semantic entropy production (SEP) metric in answer generation, and show that high faithfulness generally implies low entropy production. The SF and SEP metrics can be used jointly or separately for LLM evaluation and hallucination control. We demonstrate our framework on LLM summarization of corporate SEC 10-K filings.

Problem

• Proposes unsupervised metrics to evaluate LLM faithfulness to tasks.
• Models LLMs as information engines using thermodynamic and information theory.
• Measures semantic faithfulness and entropy to control hallucinations.

Innovation

• Unsupervised metrics using information theory and thermodynamics
• Semantic faithfulness measured via KL divergence of topic matrices
• Thermodynamics-based entropy production metric for hallucination control