Proper Calibeating

2026-05-26

AI Summary

This work unifies the notions of proper-calibration and proper-calibeating for any bounded proper scoring rule, generalizing classical calibration and calibeating to a broader setting. By integrating the theory of proper scoring rules, uniform convergence analysis, and the framework of online no-regret learning, it establishes—for the first time—an equivalence between these two properties and generic no-regret strategies. The main contributions include proving that classical calibration implies proper-calibration, devising an effective method to achieve proper-calibeating, and uncovering the theoretical foundation through which proper-calibration enhances the reliability of predictions in decision-making under uncertainty.

Abstract

The classic concept of "calibrated forecasts" and its more recent refinement, "calibeating," are defined with respect to the standard quadratic scoring rule. We extend these notions to the class of $\textit{proper}$ scoring rules (for which the best forecast is the true distribution) and define $\textit{proper-calibration}$ and $\textit{proper-calibeating}$ by requiring the errors to converge to zero uniformly over all bounded proper scoring rules. We first establish that calibration always implies proper-calibration, whereas calibeating need not imply proper-calibeating. Second, we show how to guarantee proper-calibeating and proper-multicalibeating. Finally, we demonstrate the equivalence between proper-calibration and universal no regret when best replying to forecasts in decision-making under uncertainty.

Problem

• proper scoring rules
• calibration
• calibeating
• forecasting
• decision-making under uncertainty

Innovation

• proper scoring rules
• proper-calibration
• proper-calibeating
• universal no regret
• multicalibeating