High dimensional online calibration in polynomial time

2025-04-12

AI Summary

In online probabilistic forecasting, traditional asymptotically calibrated algorithms suffer from the curse of dimensionality, requiring exponential time—exp(d)—to achieve nontrivial calibration in d dimensions. Method: We propose the first polynomial-time ε-calibration algorithm, resolving a long-standing open problem posed by Abernethy & Mannor (2011) and Hazan & Kakade (2012). Our approach leverages recent advances in swap regret minimization, employing a randomized switching strategy among sub-predictors, each outputting a distribution based on empirical frequencies over a sliding time window. Contribution/Results: The algorithm achieves ε-calibration within T = d^{O(1/ε²)} rounds; we further establish a matching lower bound of d^{Ω(log(1/ε))}, proving tightness and theoretical optimality. The method is simple, computationally efficient, and fully resolves the computational feasibility bottleneck for high-dimensional online calibration.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Probabilistic Programming
• Search and Optimization: Algorithm Configuration

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

In online (sequential) calibration, a forecaster predicts probability distributions over a finite outcome space $[d]$ over a sequence of $T$ days, with the goal of being calibrated. While asymptotically calibrated strategies are known to exist, they suffer from the curse of dimensionality: the best known algorithms require $exp(d)$ days to achieve non-trivial calibration. In this work, we present the first asymptotically calibrated strategy that guarantees non-trivial calibration after a polynomial number of rounds. Specifically, for any desired accuracy $epsilon>0$, our forecaster becomes $epsilon$-calibrated after $T = d^{O(1/epsilon^2)}$ days. We complement this result with a lower bound, proving that at least $T = d^{Omega(log(1/epsilon))}$ rounds are necessary to achieve $epsilon$-calibration. Our results resolve the open questions posed by [Abernethy-Mannor'11, Hazan-Kakade'12]. Our algorithm is inspired by recent breakthroughs in swap regret minimization [Peng-Rubinstein'24, Dagan et al.'24]. Despite its strong theoretical guarantees, the approach is remarkably simple and intuitive: it randomly selects among a set of sub-forecasters, each of which predicts the empirical outcome frequency over recent time windows.

Problem

• Develops polynomial-time online calibration for high dimensions
• Resolves curse of dimensionality in sequential forecasting
• Provides tight bounds for achieving ε-calibration efficiently

Innovation

• Polynomial-time online calibration strategy
• Random selection among sub-forecasters
• Empirical outcome frequency prediction