A Symplectic Analysis of Alternating Mirror Descent

2024-05-06

arXiv.org

AI Summary

This paper investigates the dynamics of Alternating Mirror Descent (AMD) in bilinear zero-sum games. From a Hamiltonian mechanics perspective, we discretize the continuous Hamiltonian flow using the symplectic Euler method, establishing an analytical framework that unifies symplectic geometry, Lie algebra, and convex optimization. We derive, for the first time, a closed-form expression for the modified Hamiltonian (MH) associated with AMD, revealing its fundamental distinction from previously known conserved quantities. Leveraging a novel bound on the MH truncation error, we prove that AMD achieves total regret of $O(K^{1/5})$ and duality gap of $O(K^{-4/5})$, substantially improving upon prior guarantees. Furthermore, we propose a key conjecture: under higher-order structural conditions on the MH, AMD attains near-optimal convergence—namely, $O(1)$ regret and $O(1/K)$ duality gap—matching theoretical lower bounds.

Technology Category

• Multiagent Systems: Mechanism Design
• Game Theory and Economic Paradigms: Mechanism Design
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Motivated by understanding the behavior of the Alternating Mirror Descent (AMD) algorithm for bilinear zero-sum games, we study the discretization of continuous-time Hamiltonian flow via the symplectic Euler method. We provide a framework for analysis using results from Hamiltonian dynamics, Lie algebra, and symplectic numerical integrators, with an emphasis on the existence and properties of a conserved quantity, the modified Hamiltonian (MH), for the symplectic Euler method. We compute the MH in closed-form when the original Hamiltonian is a quadratic function, and show that it generally differs from the other conserved quantity known previously in that case. We derive new error bounds on the MH when truncated at orders in the stepsize in terms of the number of iterations, $K$, and use these bounds to show an improved $mathcal{O}(K^{1/5})$ total regret bound and an $mathcal{O}(K^{-4/5})$ duality gap of the average iterates for AMD. Finally, we propose a conjecture which, if true, would imply that the total regret for AMD scales as $mathcal{O}left(K^{varepsilon} ight)$ and the duality gap of the average iterates as $mathcal{O}left(K^{-1+varepsilon} ight)$ for any $varepsilon>0$, and we can take $varepsilon=0$ upon certain convergence conditions for the MH.

Problem

• Analyzes Alternating Mirror Descent for bilinear zero-sum games
• Studies symplectic Euler discretization of Hamiltonian flow
• Derives improved regret bounds and duality gap estimates

Innovation

• Symplectic Euler method for Hamiltonian flow discretization
• Closed-form modified Hamiltonian for quadratic functions
• Improved regret bounds via truncated modified Hamiltonian