Unsupervised Confidence Calibration for Reasoning LLMs from a Single Generation

2026-04-21

AI Summary

This work addresses the challenge that large language models often produce poorly calibrated confidence estimates in single-generation settings, with existing approaches typically requiring labeled data or multiple generations, thereby limiting practical applicability. The authors propose an unsupervised calibration method that constructs a self-consistency–based proxy target offline from unlabeled data and distills it into a lightweight confidence predictor. This approach achieves reliable confidence calibration without labels and using only a single model generation—setting a new precedent in the field. Evaluated across five mathematical and question-answering benchmarks and nine reasoning models, the method significantly outperforms baseline techniques, enhances performance in selective prediction and downstream decision-making tasks, and demonstrates strong out-of-distribution generalization capabilities.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: (Large) Language Models
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Reasoning language models can solve increasingly complex tasks, but struggle to produce the calibrated confidence estimates necessary for reliable deployment. Existing calibration methods usually depend on labels or repeated sampling at inference time, making them impractical in many settings. We introduce a method for unsupervised confidence calibration of reasoning LLMs when only a single generation is available at inference time. Our approach uses offline sampling on unlabeled data to derive a self-consistency-based proxy target, then distills this signal into a lightweight deployment-time confidence predictor. In a broad evaluation across 5 math and question-answering tasks using 9 reasoning models, our method substantially outperforms baselines, including under distribution shift, and improves downstream performance in selective prediction and simulated downstream decision-making.

Problem

• confidence calibration
• reasoning LLMs
• unsupervised learning
• single generation
• reliable deployment

Innovation

• unsupervised calibration
• single-generation inference
• self-consistency
• confidence distillation
• reasoning LLMs