On the Convergence of Multicalibration Gradient Boosting

2026-02-06

AI Summary

This work establishes the first convergence theory for multi-calibrated gradient boosting under squared loss regression, addressing a long-standing gap between its widespread practical use and theoretical understanding. Assuming smoothness of the weak learners, the authors prove linear convergence of the base algorithm and further reveal local quadratic convergence for an adaptive variant. By integrating a rescaling strategy and an adaptive update mechanism within the multi-calibration framework, the analysis shows that the magnitude of prediction updates decays at a rate of \(O(1/\sqrt{T})\), while the multi-calibration error converges simultaneously. Empirical evaluations on real-world datasets corroborate both the theoretical guarantees and the rapid convergence behavior of the proposed algorithms.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Multicalibration gradient boosting has recently emerged as a scalable method that empirically produces approximately multicalibrated predictors and has been deployed at web scale. Despite this empirical success, its convergence properties are not well understood. In this paper, we bridge the gap by providing convergence guarantees for multicalibration gradient boosting in regression with squared-error loss. We show that the magnitude of successive prediction updates decays at $O(1/\sqrt{T})$, which implies the same convergence rate bound for the multicalibration error over rounds. Under additional smoothness assumptions on the weak learners, this rate improves to linear convergence. We further analyze adaptive variants, showing local quadratic convergence of the training loss, and we study rescaling schemes that preserve convergence. Experiments on real-world datasets support our theory and clarify the regimes in which the method achieves fast convergence and strong multicalibration.

Problem

• multicalibration
• gradient boosting
• convergence
• regression
• squared-error loss

Innovation

• multicalibration
• gradient boosting
• convergence analysis
• adaptive boosting
• rescaling schemes