Truthful Calibration Measures for Sequential Prediction

2026-08-21

AI Summary

本文探讨了在序列预测中实现精确真实校准的不可能性，并提出了一种近似真实性的校准方法，改进了先前的研究结果。

Abstract

Calibration requires probabilistic reports to be conditionally unbiased and reliably interpretable as probabilities. A calibration measure assigns numerical error to miscalibrated reports. Haghtalab et al. (2024) proposed an approximately truthful calibration measure for online prediction, leaving open whether exact truthfulness is compatible with completeness and soundness. We resolve this question negatively for sequential binary prediction: exact truthfulness is incompatible with completeness and soundness, even for independent outcomes. We then show that this impossibility is specific to exact truthfulness. We give two general reductions from a base calibration measure, producing additively and multiplicatively approximately truthful calibration measures, respectively. Applying the multiplicative reduction, for every $0 < \varepsilon < 1$ we construct a sound and complete calibration measure that is $(1+\exp(-T^{(1-\varepsilon)/2}/2))$-multiplicatively truthful. This improves the approximate-truthfulness guarantee of Haghtalab et al. (2024).

Problem

• truthful calibration
• sequential prediction
• completeness
• soundness

Innovation

• exact truthfulness
• completeness and soundness
• approximate truthful calibration measures
• multiplicative reduction