Equilibrium Residuals Expose Three Regimes of Matrix-Game Strategic Reasoning in Language Models

2026-05-11

AI Summary

This work addresses the limitation of large language models (LLMs) in performing genuine strategic reasoning when semantic cues are absent, as they often rely on memorization rather than equilibrium computation. By leveraging procedurally generated zero-sum matrix games, the study disentangles three underlying mechanisms: semantic recall, approximate Nash equilibrium computation, and output bottlenecks. The authors propose a residual-based training method that exploits the Lipschitz continuity of equilibrium residuals with respect to payoff perturbations, thereby circumventing the discontinuity inherent in conventional equilibrium selectors. Through a combination of procedural game generation, supervised fine-tuning, residual reinforcement learning, and dominance-action augmentation, the approach elevates the success rate on unseen 5×5 to 7×7 games from 2% to 61%. Furthermore, experiments on embedded 3×3 games demonstrate that the model acquires authentic strategic reasoning capabilities.

Abstract

Large language models can score well on named game-theory benchmarks while failing on the same strategic computation once semantic cues are removed. We show this gap with procedurally generated zero-sum matrix games: a model that recognizes familiar games drops to 34%, 18%, and 2% success on anonymous $2{\times}2$, $3{\times}3$, and $5{\times}5$ payoff matrices. The benchmark separates semantic recall, learned approximate Nash computation, and an output-interface bottleneck that limits scale. Training only on $2{\times}2$ and $3{\times}3$ games, supervised fine-tuning raises unseen $5{\times}5$--$7{\times}7$ success from 2% to 61%, while exploitability-reward training averages 37% with high seed variance. We prove that the exploitability residual is $2$-Lipschitz in payoff perturbations, unlike discontinuous vertex-returning LP equilibrium selectors, explaining why residual training can transfer under payoff shifts even when formatting instability limits mean performance. A dominated-action padding experiment provides causal evidence: trained models solve $3{\times}3$ games embedded in much larger matrices, while random-padded controls fail and dense $12{\times}12$ games remain near failure. Procedural evaluation is therefore necessary for measuring strategic reasoning, and residual rewards expose a real but format-limited route to approximate equilibrium computation.

Problem

• strategic reasoning
• matrix games
• equilibrium computation
• language models
• semantic cues

Innovation

• equilibrium residuals
• procedural evaluation
• strategic reasoning
• Lipschitz continuity
• exploitability-reward training