From token probabilities to calibrated confidence: An empirical study of mathematical question answering

2026-08-07

AI Summary

This work addresses the overconfidence of large language models in mathematical question answering, which often leads to unreliable confidence estimates. The authors systematically investigate token-probability–based calibration methods, proposing a single-pass confidence estimation approach and a low-cost in-place self-verification strategy, complemented by post-hoc techniques such as Platt scaling and isotonic regression. Experimental results demonstrate that aggregating probabilities over the full output sequence effectively discriminates between correct and incorrect answers, and that multi-channel methods substantially reduce in-domain calibration error. However, calibration performance is sensitive to problem difficulty and exhibits limited transferability across different models and datasets.

Abstract

Confidence estimation for large language models (LLMs) aims to estimate the probability that a generated answer is correct, while calibration aligns these estimates with empirical accuracy. Prior work has shown that token probabilities are often overconfident, we investigate whether these readily available signals can nevertheless provide well-calibrated confidence estimation for mathematical question answering. We compare single-pass estimators, which reuse token probabilities from the original generation, with multi-pass estimators, which obtain additional confidence signals through verification or stochastic forward passes. While individual token probabilities can be highly saturated, we find that aggregating token probabilities over the full sequence captures small but consistent differences between correct and incorrect generations, yielding more informative confidence estimates. Multi-pass methods can yield calibrated confidence estimates. We study two such approaches: self-verification through re-prompting, including a lower-cost in-situ variant, and Monte Carlo Dropout, which derives confidence from variation across stochastic forward passes. We further evaluate two post-hoc calibration methods, Platt scaling and isotonic regression, both of which substantially reduce in-domain calibration error. However, their data efficiency varies with dataset difficulty, and the calibration mappings often transfer asymmetrically across datasets and models.

Problem

• confidence estimation
• calibration
• mathematical question answering
• large language models
• token probabilities

Innovation

• confidence estimation
• calibration
• token probabilities
• mathematical question answering
• Monte Carlo Dropout