ADVICE: Answer-Dependent Verbalized Confidence Estimation

2025-10-12

AI Summary

Large language models (LLMs) commonly exhibit overconfidence and answer-agnostic confidence estimation in natural language confidence expression, leading to poor calibration and diminished reliability. This work identifies the lack of semantic dependency between confidence statements and generated answers as the root cause. To address this, we propose ADVICE—a novel Answer-Dependent Confidence Estimation framework—that enforces strong semantic coupling between natural language confidence expressions and answer content. ADVICE achieves this through fine-grained supervised fine-tuning and a dynamic confidence alignment mechanism. Extensive experiments across multiple benchmarks demonstrate that ADVICE significantly improves confidence calibration—reducing Expected Calibration Error (ECE) by 32–58%—while preserving original task accuracy. Unlike prior methods, ADVICE yields more balanced, interpretable, and reliable self-assessment without compromising performance, establishing a new paradigm for answer-aware confidence modeling in LLMs.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Safety and Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Recent progress in large language models (LLMs) has enabled them to express their confidence in natural language, enhancing transparency and reliability. However, their confidence often exhibits overconfidence, the cause of which remains poorly understood. In this work, we conduct a detailed analysis of the dynamics underlying verbalized confidence and identify answer-independence as a key factor, defined as the model's failure to condition confidence on its own answer. To address this, we propose ADVICE (Answer-Dependent Verbalized Confidence Estimation), a fine-tuning framework that facilitates answer-grounded confidence estimation. Extensive experiments show that ADVICE substantially improves confidence calibration while preserving task performance. Further analyses confirm that ADVICE strengthens answer-groundedness, leading to more balanced and well-calibrated confidence distributions. Our findings shed light on the origin of overconfidence and establish a framework for more trustworthy confidence verbalization.

Problem

• Addresses overconfidence in LLMs' verbalized confidence estimates
• Identifies answer-independence as key cause of poor calibration
• Proposes fine-tuning framework for answer-dependent confidence estimation

Innovation

• Fine-tuning framework for answer-grounded confidence estimation
• Improves confidence calibration while preserving task performance
• Strengthens answer-groundedness for balanced confidence distributions