Accelerated and Stable Convergence with Anchored Optimistic Method

2026-06-19

AI Summary

This work addresses the limitation in online and stochastic minimax optimization of monotone variational inequalities, where existing methods typically require two gradient evaluations per iteration. To overcome this, the paper proposes the Generalized Optimistic Method with Anchoring (GOMA), which integrates a two-timescale optimistic update with an anchoring mechanism inspired by Halpern iterations, requiring only a single gradient query per round. Theoretically, in the deterministic setting, GOMA achieves—for the first time—an optimal $O(1/k^2)$ convergence rate in terms of the squared gradient norm. Moreover, in the unconstrained, stochastic, monotone Lipschitz setting without variance reduction or increasing batch sizes, GOMA provides the first $O(1/\sqrt{k})$ convergence guarantee for the last iterate.

Abstract

We study first-order methods for solving monotone variational inequalities arising in min-max optimization. Classical approaches such as the extragradient method rely on two gradient queries per iteration, which limits their analysis and applicability in the online and stochastic settings. We propose a family of Generalized Optimistic Methods with Anchoring (GOMA), which combine two-time-scale optimistic updates with an anchoring term inspired by Halpern iteration. In the deterministic setting, GOMA achieves the optimal accelerated last-iterate rate $O(1/k^2)$ on the squared gradient norm for monotone Lipschitz operators. In the stochastic setting with unbounded variance, a simplified single-call variant of GOMA achieves a last-iterate convergence rate of $O(1/\sqrt{k})$ on the squared gradient norm. To the best of our knowledge, this is the first such guarantee for stochastic monotone Lipschitz variational inequalities in the unconstrained setting without variance reduction or growing batches.

Problem

• monotone variational inequalities
• min-max optimization
• stochastic setting
• last-iterate convergence
• first-order methods

Innovation

• Generalized Optimistic Methods with Anchoring
• last-iterate convergence
• monotone variational inequalities
• accelerated convergence
• stochastic optimization