Boosting Perturbed Gradient Ascent for Last-Iterate Convergence in Games

2024-10-03

arXiv.org

AI Summary

This work addresses the challenge of achieving last-iterate convergence for first-order algorithms in monotone games under additive payoff noise. We propose an enhanced payoff-perturbed gradient ascent method. Our core innovations are a boosting-type perturbation mechanism and a periodic anchor-point reset strategy, which jointly enforce strong convexity via perturbation and enable analysis through monotone operator theory—without requiring precise hyperparameter tuning. Theoretically, the method achieves an $O(1/T)$ last-iterate convergence rate under stochastic noise, strictly improving upon existing perturbation-based approaches. Empirical results demonstrate its robustness and practical effectiveness across benchmark game settings.

Technology Category

• Game Theory and Economic Paradigms: Mechanism Design
• Multiagent Systems: Mechanism Design
• Machine Learning: Online Learning & Bandits

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

This paper presents a payoff perturbation technique, introducing a strong convexity to players' payoff functions in games. This technique is specifically designed for first-order methods to achieve last-iterate convergence in games where the gradient of the payoff functions is monotone in the strategy profile space, potentially containing additive noise. Although perturbation is known to facilitate the convergence of learning algorithms, the magnitude of perturbation requires careful adjustment to ensure last-iterate convergence. Previous studies have proposed a scheme in which the magnitude is determined by the distance from a periodically re-initialized anchoring or reference strategy. Building upon this, we propose Gradient Ascent with Boosting Payoff Perturbation, which incorporates a novel perturbation into the underlying payoff function, maintaining the periodically re-initializing anchoring strategy scheme. This innovation empowers us to provide faster last-iterate convergence rates against the existing payoff perturbed algorithms, even in the presence of additive noise.

Problem

• Achieving last-iterate convergence in games with monotone gradient payoff functions.
• Adjusting perturbation magnitude to ensure convergence in noisy environments.
• Improving convergence rates with a novel boosting payoff perturbation technique.

Innovation

• Introduces payoff perturbation for convexity
• Uses periodic re-initialized anchoring strategy
• Enhances convergence rates with novel perturbation