On the Stability Connection Between Discrete-Time Algorithms and Their Resolution ODEs: Applications to Min-Max Optimisation

2026-03-01

AI Summary

This work investigates the stability correspondence between discrete-time min-max optimization algorithms and their high-order (O(s^r)) resolution ordinary differential equations (ODEs). By constructing a high-order resolution ODE framework, the study establishes—without assuming Hessian invariance—a direct analysis of exponential stability for the continuous dynamics, thereby rigorously linking the saddle-point stability of several algorithms, including TT-GDA, GEG, TT-PPM, and DN, to their associated continuous dynamical systems. Theoretical analysis demonstrates that, under appropriate hyperparameter choices, the saddle points of these algorithms form exponentially stable equilibrium sets. Numerical experiments further corroborate these findings, significantly extending the applicability of stability theory in nonconvex optimization settings.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

This work establishes a rigorous connection between stability properties of discrete-time algorithms (DTAs) and corresponding continuous-time dynamical systems derived through $ O(s^r) $-resolution ordinary differential equations (ODEs). We show that for discrete- and continuous-time dynamical systems satisfying a mild error assumption, exponential stability of a common equilibrium with respect to the continuous time dynamics implies exponential stability of the corresponding equilibrium for the discrete-time dynamics, provided that the step size is chosen sufficiently small. We extend this result to common compact invariant sets. We prove that if an equilibrium is exponentially stable for the $ O(s^r) $-resolution ODE, then it is also exponentially stable for the associated DTA. We apply this framework to analyse the limit point properties of several prominent optimisation algorithms, including Two-Timescale Gradient Descent--Ascent (TT-GDA), Generalised Extragradient (GEG), Two-Timescale Proximal Point (TT-PPM), Damped Newton (DN), Regularised Damped Newton (RDN), and the Jacobian method (JM), by studying their $ O(1) $- and $ O(s) $-resolution ODEs. We show that under a proper choice of hyperparameters, the set of saddle points of the objective function is a subset of the set of exponentially stable equilibria of GEG, TT-PPM, DN, and RDN. We relax the common Hessian invariance assumption through direct analysis of the resolution ODEs, broadening the applicability of our results. Numerical examples illustrate the theoretical findings.

Problem

• stability
• discrete-time algorithms
• resolution ODEs
• min-max optimisation
• exponential stability

Innovation

• resolution ODEs
• exponential stability
• discrete-time algorithms
• min-max optimization
• invariant sets