Efficient Swap Multicalibration of Elicitable Properties

2025-11-07

AI Summary

This work studies multicalibration with respect to an elicitable property Γ, generalizing classical group-based calibration to arbitrary bounded hypothesis classes and introducing the stronger notion of *swap multicalibration*. For ℓᵣ-loss (r ≥ 2), we design the first oracle-efficient online algorithm—requiring only access to a standard online agnostic learner—that achieves, with high probability, a calibration error bound of O(T^{1/(r+1)}). Notably, for r = 2, it attains O(T^{1/3}) ℓ₂-swap multicalibration error, breaking the prior Ω(√T) barrier and resolving the long-standing open problem of efficient ℓ₂-mean multicalibration. Technically, our approach integrates tools from online learning, sequential Rademacher complexity analysis, and identifiability theory, yielding a more general and tighter theoretical and algorithmic framework for conditional accuracy guarantees in fair machine learning.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Multicalibration [HJKRR18] is an algorithmic fairness perspective that demands that the predictions of a predictor are correct conditional on themselves and membership in a collection of potentially overlapping subgroups of a population. The work of [NR23] established a surprising connection between multicalibration for an arbitrary property $Gamma$ (e.g., mean or median) and property elicitation: a property $Gamma$ can be multicalibrated if and only if it is elicitable, where elicitability is the notion that the true property value of a distribution can be obtained by solving a regression problem over the distribution. In the online setting, [NR23] proposed an inefficient algorithm that achieves $sqrt T$ $ell_2$-multicalibration error for a hypothesis class of group membership functions and an elicitable property $Gamma$, after $T$ rounds of interaction between a forecaster and adversary. In this paper, we generalize multicalibration for an elicitable property $Gamma$ from group membership functions to arbitrary bounded hypothesis classes and introduce a stronger notion -- swap multicalibration, following [GKR23]. Subsequently, we propose an oracle-efficient algorithm which, when given access to an online agnostic learner, achieves $T^{1/(r+1)}$ $ell_r$-swap multicalibration error with high probability (for $rge2$) for a hypothesis class with bounded sequential Rademacher complexity and an elicitable property $Gamma$. For the special case of $r=2$, this implies an oracle-efficient algorithm that achieves $T^{1/3}$ $ell_2$-swap multicalibration error, which significantly improves on the previously established bounds for the problem [NR23, GMS25, LSS25a], and completely resolves an open question raised in [GJRR24] on the possibility of an oracle-efficient algorithm that achieves $sqrt{T}$ $ell_2$-mean multicalibration error by answering it in a strongly affirmative sense.

Problem

• Generalizing multicalibration from groups to arbitrary hypothesis classes
• Achieving stronger swap multicalibration for elicitable properties efficiently
• Improving error bounds for oracle-efficient multicalibration algorithms

Innovation

• Generalizes multicalibration to arbitrary bounded hypothesis classes
• Introduces swap multicalibration for elicitable properties
• Proposes oracle-efficient algorithm with improved error bounds