Proximal Regret and Proximal Correlated Equilibria: A New Tractable Solution Concept for Online Learning and Games

2025-11-03

AI Summary

This paper addresses the unification challenge between learning dynamics and equilibrium computation in algorithmic game theory. We introduce *proximal regret*, a novel regret measure situated between external and swap regret, and define the corresponding *proximal correlated equilibrium* (PCE). We prove that when all players employ no-proximal-regret algorithms, their joint empirical distribution converges to a PCE. Further, we derive proximal regret bounds for online gradient descent, mirror descent, and optimistic gradient descent (OGD) in convex games under Bregman geometries—revealing, for the first time, that gradient-based methods implicitly optimize this stronger regret notion. Notably, OGD achieves faster convergence in smooth convex games, providing new theoretical justification for its empirical superiority. Our framework unifies emerging equilibrium concepts—including gradient equilibria and semi-coarse correlated equilibria—and extends the expressive power of equilibrium modeling.

Technology Category

• Game Theory and Economic Paradigms: Equilibrium
• Machine Learning: Online Learning & Bandits
• Multiagent Systems: Mechanism Design

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Learning and computation of equilibria are central problems in algorithmic game theory. In this work, we introduce proximal regret, a new notion of regret based on proximal operators that lies strictly between external and swap regret. When every player employs a no-proximal-regret algorithm in a general convex game, the empirical distribution of play converges to proximal correlated equilibria (PCE), a refinement of coarse correlated equilibria. Our framework unifies several emerging notions in online learning and game theory -- such as gradient equilibrium and semicoarse correlated equilibrium -- and introduces new ones. Our main result shows that the classic Online Gradient Descent (GD) algorithm achieves an optimal $O(sqrt{T})$ bound on proximal regret, revealing that GD, without modification, minimizes a stronger regret notion than external regret. This provides a new explanation for the empirically superior performance of gradient descent in online learning and games. We further extend our analysis to Mirror Descent in the Bregman setting and to Optimistic Gradient Descent, which yields faster convergence in smooth convex games.

Problem

• Introduces proximal regret as a new solution concept for online learning
• Establishes convergence to proximal correlated equilibria in convex games
• Shows classic gradient descent achieves optimal proximal regret bounds

Innovation

• Proximal regret bridges external and swap regret
• Online Gradient Descent achieves optimal proximal regret bound
• Framework unifies gradient equilibrium and correlated equilibrium