The Hamiltonian of Poly-matrix Zero-sum Games

2025-05-19

AI Summary

This work addresses the lack of structured physical interpretation and last-iterate convergence guarantees for learning dynamics in multi-matrix zero-sum games. We establish, for the first time, a Hamiltonian mechanics framework for such games: strategies and cumulative payoffs are modeled as conjugate variables, yielding an explicit Hamiltonian function that reveals intrinsic probability conservation and Fenchel coupling invariance. Building on this, we propose Dissipative Follow-the-Regularized-Leader (DFTRL), a physically grounded dynamical system incorporating directed dissipation to break conservative behavior. We prove that DFTRL achieves global convergence to Nash equilibria in multi-matrix zero-sum games, with guaranteed last-iterate convergence. This framework unifies the geometric and physical interpretations of diverse convergent algorithms and introduces a novel Hamiltonian analytical paradigm for game-theoretic learning.

Technology Category

• Game Theory and Economic Paradigms: Adversarial Learning
• Multiagent Systems: Multiagent Learning
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Understanding a dynamical system fundamentally relies on establishing an appropriate Hamiltonian function and elucidating its symmetries. By formulating agents' strategies and cumulative payoffs as canonically conjugate variables, we identify the Hamiltonian function that generates the dynamics of poly-matrix zero-sum games. We reveal the symmetries of our Hamiltonian and derive the associated conserved quantities, showing how the conservation of probability and the invariance of the Fenchel coupling are intrinsically encoded within the system. Furthermore, we propose the dissipation FTRL (DFTRL) dynamics by introducing a perturbation that dissipates the Fenchel coupling, proving convergence to the Nash equilibrium and linking DFTRL to last-iterate convergent algorithms. Our results highlight the potential of Hamiltonian dynamics in uncovering the structural properties of learning dynamics in games, and pave the way for broader applications of Hamiltonian dynamics in game theory and machine learning.

Problem

• Identifies Hamiltonian function for poly-matrix zero-sum games dynamics
• Reveals symmetries and conserved quantities in game Hamiltonian
• Proposes DFTRL dynamics for Nash equilibrium convergence

Innovation

• Formulate strategies and payoffs as conjugate variables
• Reveal symmetries and derive conserved quantities
• Propose DFTRL dynamics for Nash equilibrium convergence