Closing the Confidence-Faithfulness Gap in Large Language Models

2026-03-26

AI Summary

This work addresses the frequent misalignment between large language models’ stated confidence and their actual accuracy—a manifestation of poor calibration. Through mechanistic interpretability analysis, we discover that internal calibration signals and verbalized confidence signals are approximately orthogonal in the model’s linear representation space. We further identify a novel “reasoning contamination effect,” wherein the reasoning process interferes with faithful confidence expression. To mitigate this issue, we propose a two-stage adaptive prompting framework that integrates linear probing with Contrastive Activation Addition (CAA). Evaluated across three open-source large language models and four benchmark datasets, our approach significantly improves the consistency between expressed confidence and factual accuracy, effectively narrowing the gap between confidence and faithfulness.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Safety and Robustness
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Large language models (LLMs) tend to verbalize confidence scores that are largely detached from their actual accuracy, yet the geometric relationship governing this behavior remain poorly understood. In this work, we present a mechanistic interpretability analysis of verbalized confidence, using linear probes and contrastive activation addition (CAA) steering to show that calibration and verbalized confidence signals are encoded linearly but are orthogonal to one another -- a finding consistent across three open-weight models and four datasets. Interestingly, when models are prompted to simultaneously reason through a problem and verbalize a confidence score, the reasoning process disrupts the verbalized confidence direction, exacerbating miscalibration. We term this the "Reasoning Contamination Effect." Leveraging this insight, we introduce a two-stage adaptive steering pipeline that reads the model's internal accuracy estimate and steers verbalized output to match it, substantially improving calibration alignment across all evaluated models.

Problem

• confidence-faithfulness gap
• large language models
• calibration
• verbalized confidence
• reasoning contamination

Innovation

• mechanistic interpretability
• confidence calibration
• contrastive activation addition
• reasoning contamination effect
• adaptive steering