The Oracle Complexity of Simplex-based Matrix Games: Linear Separability and Nash Equilibria

2024-12-09

arXiv.org

AI Summary

This paper studies the oracle complexity of matrix games $max_{mathbf{w}inmathcal{W}}min_{mathbf{p}inDelta}mathbf{p}^ op Amathbf{w}$ under simplex constraints, focusing on linear separability testing and $varepsilon$-approximate Nash equilibrium computation in zero-sum games. Within the Nemirovski–Yudin oracle framework, we establish the first tight lower bounds for both single- and double-sided matrix-vector multiplication oracles. For linear separability with margin $gamma_a$, we prove an $Omega(gamma_a^{-2})$ lower bound for single-sided queries—matching the Perceptron algorithm’s rate—and a $widetilde{Omega}(gamma_a^{-2/3})$ lower bound under double-sided access, demonstrating inherent limitations of acceleration. For Nash equilibrium approximation, we improve the prior best lower bound to $widetilde{Omega}(varepsilon^{-2/5})$, achieving an exponential improvement over previous results. Our key technical innovation is an $ell_1$-geometry-adapted analysis, which enables the first tight oracle complexity lower bounds for double-sided matrix multiplication.

Technology Category

• Search and Optimization: Non-convex Optimization
• Game Theory and Economic Paradigms: Adversarial Learning
• Machine Learning: Online Learning & Bandits

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

We study the problem of solving matrix games of the form $max_{mathbf{w}inmathcal{W}}min_{mathbf{p}inDelta}mathbf{p}^{ op}Amathbf{w}$, where $A$ is some matrix and $Delta$ is the probability simplex. This problem encapsulates canonical tasks such as finding a linear separator and computing Nash equilibria in zero-sum games. However, perhaps surprisingly, its inherent complexity (as formalized in the standard framework of oracle complexity [Nemirovski and Yudin, 1983]) is not well-understood. In this work, we first identify different oracle models which are implicitly used by prior algorithms, amounting to multiplying the matrix $A$ by a vector from either one or both sides. We then prove complexity lower bounds for algorithms under both access models, which in particular imply a separation between them. Specifically, we start by proving that algorithms for linear separability based on one-sided multiplications must require $Omega(gamma_A^{-2})$ iterations, where $gamma_A$ is the margin, as matched by the Perceptron algorithm. We then prove that accelerated algorithms for this task, which utilize multiplications from both sides, must require $ ilde{Omega}(gamma_{A}^{-2/3})$ iterations, establishing the first oracle complexity barrier for such algorithms. Finally, by adapting our lower bound to $ell_1$ geometry, we prove that computing an $epsilon$-approximate Nash equilibrium requires $ ilde{Omega}(epsilon^{-2/5})$ iterations, which is an exponential improvement over the previously best-known lower bound due to Hadiji et al. [2024].

Problem

• Study oracle complexity of simplex-based matrix games
• Prove lower bounds for one-sided and two-sided oracle models
• Establish iteration limits for linear separability and Nash equilibria

Innovation

• Oracle models for matrix multiplication complexity
• Lower bounds for one-sided multiplication algorithms
• Improved Nash equilibrium computation complexity