Future Confidence Distillation in Large Language Models

2026-07-08

AI Summary

Current large language models typically estimate confidence based on complete responses, which hinders their applicability in reliability-sensitive real-time decision-making scenarios. This work proposes a method termed "future confidence distillation," which leverages high-accuracy posterior confidence—computed after answer generation—as a teacher signal to train a lightweight predictor that operates solely on pre-generation hidden states. By analyzing the linearly decodable confidence information embedded in these representations and applying knowledge distillation, the approach enables accurate and efficient confidence estimation prior to response completion. The resulting model maintains excellent calibration while demonstrating strong sample efficiency and robust generalization across diverse datasets.

Abstract

Reliable confidence estimation is essential for deploying large language models (LLMs) in confidence-aware systems, where downstream decisions such as retrieval, tool use, and adaptive computation depend on accurately estimating answer reliability. Existing approaches, however, largely treat confidence as a property of completed responses, overlooking how confidence-related information evolves throughout the answering process. In this work, we investigate confidence from a temporal perspective by comparing pre-solution Feeling-of-Knowing (FOK) and post-solution Judgement-of-Learning (JOL) confidence estimates across frontier and open-source LLMs. We show that post-solution confidence is consistently better calibrated and more discriminative than pre-solution confidence, while linear probes trained on hidden representations recover substantially richer confidence-related information than models explicitly verbalise. Building on this observation, we introduce future confidence distillation, which trains predictors operating on pre-solution hidden representations using teacher confidence estimates produced by post-solution correctness probes. Despite requiring only pre-solution representations for inference, distilled predictors recover much of the calibration improvement achieved by post-solution confidence, remain highly sample efficient, and transfer across datasets within the same domain. Together, our findings demonstrate that confidence-related information evolves throughout the answering process and can be anticipated before answer generation is complete, enabling significantly more reliable yet low-cost confidence estimation.

Problem

• confidence estimation
• large language models
• Feeling-of-Knowing
• Judgement-of-Learning
• temporal dynamics

Innovation

• future confidence distillation
• confidence calibration
• Feeling-of-Knowing
• Judgement-of-Learning
• hidden representations