RAMPAGE: RAndomized Mid-Point for debiAsed Gradient Extrapolation

2026-03-23

AI Summary

This work addresses the bias introduced by discretization in the extragradient (EG) method when applied to nonlinear vector fields. To mitigate this issue, the authors propose RAMPAGE and its variance-reduced variant, RAMPAGE+. These methods employ an unbiased stochastic midpoint mechanism combined with dual sampling to construct an unbiased gradient extrapolation. RAMPAGE+ further leverages negative correlation to eliminate the leading-order variance term. Under co-coercivity, co-monotonicity, and generalized Lipschitz conditions, both algorithms achieve an $O(1/k)$ convergence rate for smooth convex-concave games, applicable to both stochastic and deterministic settings. Moreover, the framework extends naturally to constrained variational inequalities, yielding the first deterministic convergence guarantees across multiple problem formulations.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Non-convex Optimization
• Multiagent Systems: Mechanism Design

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

A celebrated method for Variational Inequalities (VIs) is Extragradient (EG), which can be viewed as a standard discrete-time integration scheme. With this view in mind, in this paper we show that EG may suffer from discretization bias when applied to non-linear vector fields, conservative or otherwise. To resolve this discretization shortcoming, we introduce RAndomized Mid-Point for debiAsed Gradient Extrapolation (RAMPAGE) and its variance-reduced counterpart, RAMPAGE+ which leverages antithetic sampling. In contrast with EG, both methods are unbiased. Furthermore, leveraging negative correlation, RAMPAGE+ acts as an unbiased, geometric path-integrator that completely removes internal first-order terms from the variance, provably improving upon RAMPAGE. We further demonstrate that both methods enjoy provable $\mathcal{O}(1/k)$ convergence guarantees for a range of problems including root finding under co-coercive, co-hypomonotone, and generalized Lipschitzness regimes. Furthermore, we introduce symmetrically scaled variants to extend our results to constrained VIs. Finally, we provide convergence guarantees of both methods for stochastic and deterministic smooth convex-concave games. Somewhat interestingly, despite being a randomized method, RAMPAGE+ attains purely deterministic bounds for a number of the studied settings.

Problem

• Variational Inequalities
• Extragradient
• Discretization Bias
• Nonlinear Vector Fields
• Gradient Extrapolation

Innovation

• unbiased gradient extrapolation
• randomized midpoint
• antithetic sampling
• variance reduction
• variational inequalities