Operationalizing Stein's Method for Online Linear Optimization: CLT-Based Optimal Tradeoffs

2026-02-06

AI Summary

This work addresses the performance trade-off in adversarial online linear optimization under unknown adversary difficulty by introducing Stein’s method—previously unexplored in online learning—into algorithm design. By integrating the Wasserstein martingale central limit theorem with a dynamic programming-based surrogate strategy, the authors develop a parameter-free algorithm that achieves both an additive-tight regret bound and a tight upper bound on cumulative loss. The proposed method matches the computational complexity of Online Gradient Descent (OGD) and Multiplicative Weight Updates (MWU), while enabling continuous tuning of the optimal two-point Pareto trade-off between total loss and maximum regret. Furthermore, it provides sharp expected performance guarantees under noisy feedback, maintaining robustness without sacrificing efficiency.

Technology Category

• Machine Learning: Online Learning & Bandits
• Game Theory and Economic Paradigms: Adversarial Learning
• Search and Optimization: Adversarial Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments

Abstract

Adversarial online linear optimization (OLO) is essentially about making performance tradeoffs with respect to the unknown difficulty of the adversary. In the setting of one-dimensional fixed-time OLO on a bounded domain, it has been observed since Cover (1966) that achievable tradeoffs are governed by probabilistic inequalities, and these descriptive results can be converted into algorithms via dynamic programming, which, however, is not computationally efficient. We address this limitation by showing that Stein's method, a classical framework underlying the proofs of probabilistic limit theorems, can be operationalized as computationally efficient OLO algorithms. The associated regret and total loss upper bounds are"additively sharp", meaning that they surpass the conventional big-O optimality and match normal-approximation-based lower bounds by additive lower order terms. Our construction is inspired by the remarkably clean proof of a Wasserstein martingale central limit theorem (CLT) due to R\"ollin (2018). Several concrete benefits can be obtained from this general technique. First, with the same computational complexity, the proposed algorithm improves upon the total loss upper bounds of online gradient descent (OGD) and multiplicative weight update (MWU). Second, our algorithm can realize a continuum of optimal two-point tradeoffs between the total loss and the maximum regret over comparators, improving upon prior works in parameter-free online learning. Third, by allowing the adversary to randomize on an unbounded support, we achieve sharp in-expectation performance guarantees for OLO with noisy feedback.

Problem

• online linear optimization
• adversarial learning
• performance tradeoffs
• computational efficiency
• regret minimization

Innovation

• Stein's method
• online linear optimization
• additively sharp bounds
• martingale CLT
• parameter-free learning