How Uncertainty Estimation Scales with Sampling in Reasoning Models

2026-03-19

AI Summary

This work addresses the lack of reliable and scalable uncertainty estimation methods for reasoning language models in complex scenarios. The authors propose a black-box hybrid estimator based on parallel sampling that integrates verbalized confidence with self-consistency signals. Systematic evaluation across three prominent reasoning models and seventeen diverse tasks demonstrates that these two signals exhibit complementary strengths depending on the number of samples, with significant improvements in uncertainty calibration achievable using as few as two samples. The method yields an average AUROC gain of 12 points, achieving the strongest performance on mathematical reasoning tasks and revealing notable disparities in effectiveness between STEM and humanities domains.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Multiagent Systems: Multiagent Systems under Uncertainty

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Uncertainty estimation is critical for deploying reasoning language models, yet remains poorly understood under extended chain-of-thought reasoning. We study parallel sampling as a fully black-box approach using verbalized confidence and self-consistency. Across three reasoning models and 17 tasks spanning mathematics, STEM, and humanities, we characterize how these signals scale. Both self-consistency and verbalized confidence scale in reasoning models, but self-consistency exhibits lower initial discrimination and lags behind verbalized confidence under moderate sampling. Most uncertainty gains, however, arise from signal combination: with just two samples, a hybrid estimator improves AUROC by up to $+12$ on average and already outperforms either signal alone even when scaled to much larger budgets, after which returns diminish. These effects are domain-dependent: in mathematics, the native domain of RLVR-style post-training, reasoning models achieve higher uncertainty quality and exhibit both stronger complementarity and faster scaling than in STEM or humanities.

Problem

• uncertainty estimation
• reasoning models
• chain-of-thought
• sampling
• self-consistency

Innovation

• uncertainty estimation
• parallel sampling
• self-consistency
• verbalized confidence
• reasoning models