Calibeating Made Simple

2026-03-23

AI Summary

This work studies the problem of online post-processing of external predictions to simultaneously minimize cumulative loss and match an information-theoretic benchmark—a task known as calibeating. By establishing a minimax equivalence between calibeating and classical online learning problems such as regret minimization and prediction with expert advice, the paper embeds calibeating for the first time within the standard online learning framework. The approach applies broadly to general proper losses, mixable losses, and bounded losses, yielding a unified optimal calibeating rate. This not only recovers and extends the known $O(\log T)$ rates for Brier and logarithmic losses but also establishes new optimal rates in multi-calibration settings. Furthermore, the authors propose the first algorithm for binary prediction that simultaneously achieves calibration and an $O(\log T)$ calibeating rate.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

We study calibeating, the problem of post-processing external forecasts online to minimize cumulative losses and match an informativeness-based benchmark. Unlike prior work, which analyzed calibeating for specific losses with specific arguments, we reduce calibeating to existing online learning techniques and obtain results for general proper losses. More concretely, we first show that calibeating is minimax-equivalent to regret minimization. This recovers the $O(\log T)$ calibeating rate of Foster and Hart [FH23] for the Brier and log losses and its optimality, and yields new optimal calibeating rates for mixable losses and general bounded losses. Second, we prove that multi-calibeating is minimax-equivalent to the combination of calibeating and the classical expert problem. This yields new optimal multi-calibeating rates for mixable losses, including Brier and log losses, and general bounded losses. Finally, we obtain new bounds for achieving calibeating and calibration simultaneously for the Brier loss. For binary predictions, our result gives the first calibrated algorithm that at the same time also achieves the optimal $O(\log T)$ calibeating rate.

Problem

• calibeating
• online learning
• proper losses
• calibration
• minimax equivalence

Innovation

• calibeating
• online learning
• regret minimization
• proper losses
• calibration