Negative Momentum for Convex-Concave Optimization

2026-04-18

AI Summary

This work addresses the unclear convergence and acceleration properties of negative momentum in convex-concave minimax optimization. By combining continuous-time dynamical system analysis with discrete algorithm design, it establishes—for the first time—the global convergence of negative momentum in general convex-concave settings and demonstrates accelerated convergence under strong convexity–strong concavity. The study overcomes the prevailing belief that negative momentum is inherently difficult to stabilize, thereby validating its effectiveness across a broader class of minimax problems. The resulting algorithms achieve significantly improved convergence rates, matching or even surpassing the performance of state-of-the-art methods.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Content-based information diffusion
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

This paper revisits momentum in the context of min-max optimization. Momentum is a celebrated mechanism for accelerating gradient dynamics in settings like convex minimization, but its direct use in min-max optimization makes gradient dynamics diverge. Surprisingly, Gidel et al. 2019 showed that negative momentum can help fix convergence. However, despite these promising initial results and progress since, the power of momentum remains unclear for min-max optimization in two key ways. (1) Generality: is global convergence possible for the foundational setting of convex-concave optimization? This is the direct analog of convex minimization and is a standard testing ground for min-max algorithms. (2) Fast convergence: is accelerated convergence possible for strongly-convex-strong-concave optimization (the only non-linear setting where global convergence is known)? Recent work has even argued that this is impossible. We answer both these questions in the affirmative. Together, these results put negative momentum on more equal footing with competitor algorithms, and show that negative momentum enables convergence significantly faster and more generally than was known possible.

Problem

• min-max optimization
• convex-concave optimization
• negative momentum
• global convergence
• accelerated convergence

Innovation

• negative momentum
• min-max optimization
• convex-concave
• accelerated convergence
• global convergence