🤖 AI Summary
Regret Matching⁺ (RM⁺) and its variants—despite widespread use in two-player zero-sum games—lack last-iterate convergence guarantees, and their limit-point behavior remains poorly understood. This work first characterizes the geometric structure of RM⁺ limit points, revealing the fundamental non-convergence of standard, predictive, and alternating RM⁺ variants even in simple 3×3 matrix games.
Method: We propose two novel algorithms—Smooth Predictive RM⁺ and Extragradient RM⁺—that integrate smoothing, predictive updates, extragradient steps, and adaptive restarts.
Contribution/Results: We establish asymptotic last-iterate convergence for both algorithms, achieving the optimal sublinear rate O(1/√t); with restarts, linear convergence is attained. Leveraging non-monotone operator analysis, our convergence proofs require no restrictive rate assumptions. Numerical experiments confirm the failure of existing RM⁺ variants and demonstrate the efficacy and robustness of the proposed methods.
📝 Abstract
We study last-iterate convergence properties of algorithms for solving two-player zero-sum games based on Regret Matching$^+$ (RM$^+$). Despite their widespread use for solving real games, virtually nothing is known about their last-iterate convergence. A major obstacle to analyzing RM-type dynamics is that their regret operators lack Lipschitzness and (pseudo)monotonicity. We start by showing numerically that several variants used in practice, such as RM$^+$, predictive RM$^+$ and alternating RM$^+$, all lack last-iterate convergence guarantees even on a simple $3 imes 3$ matrix game. We then prove that recent variants of these algorithms based on a smoothing technique, extragradient RM$^{+}$ and smooth Predictive RM$^+$, enjoy asymptotic last-iterate convergence (without a rate), $1/sqrt{t}$ best-iterate convergence, and when combined with restarting, linear-rate last-iterate convergence. Our analysis builds on a new characterization of the geometric structure of the limit points of our algorithms, marking a significant departure from most of the literature on last-iterate convergence. We believe that our analysis may be of independent interest and offers a fresh perspective for studying last-iterate convergence in algorithms based on non-monotone operators.
