Efficient Uncoupled Learning Dynamics with $\tilde{O}\!\left(T^{-1/4}\right)$ Last-Iterate Convergence in Bilinear Saddle-Point Problems over Convex Sets under Bandit Feedback

2026-02-24

AI Summary

This work proposes the first decoupled learning algorithm that achieves last-iterate convergence to a Nash equilibrium with high probability in bilinear saddle-point problems under bandit feedback, where each player’s action set is a compact convex set. The method integrates experimental design with the Follow-The-Regularized-Leader (FTRL) framework, employing adaptive regularizers tailored to the geometric structure of each player’s action set and relying solely on linear optimization oracles. Despite its computational efficiency, the algorithm attains a convergence rate of $\tilde{O}(T^{-1/4})$, providing the first high-probability last-iterate guarantee in this setting.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Learning to Search
• Game Theory and Economic Paradigms: Mechanism Design

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

In this paper, we study last-iterate convergence of learning algorithms in bilinear saddle-point problems, a preferable notion of convergence that captures the day-to-day behavior of learning dynamics. We focus on the challenging setting where players select actions from compact convex sets and receive only bandit feedback. Our main contribution is the design of an uncoupled learning algorithm that guarantees last-iterate convergence to the Nash equilibrium with high probability. We establish a convergence rate of $\tilde{O}(T^{-1/4})$ up to polynomial factors in problem parameters. Crucially, our proposed algorithm is computationally efficient, requiring only an efficient linear optimization oracle over the players' compact action sets. The algorithm is obtained by combining techniques from experimental design and the classic Follow-The-Regularized-Leader (FTRL) framework, with a carefully chosen regularizer function tailored to the geometry of the action set of each learner.

Problem

• bilinear saddle-point problems
• last-iterate convergence
• bandit feedback
• Nash equilibrium
• convex sets

Innovation

• last-iterate convergence
• bilinear saddle-point problems
• bandit feedback
• uncoupled learning
• FTRL with geometry-aware regularizer