A Controllability Perspective on Steering Follow-the-Regularized-Leader Learners in Games

2026-03-27

AI Summary

This study investigates whether a single controlling agent can steer a multi-agent system to a desired state solely by selecting its own mixed strategy, without altering the underlying payoff structure, in games where all other agents follow Follow-the-Regularized-Leader (FTRL) dynamics. The authors model FTRL dynamics as a nonlinear control system on the probability simplex and, for the first time, analyze its controllability from a control-theoretic perspective. Leveraging tools from differential geometry, the Lie algebra rank condition, and projected payoff maps, they establish necessary and sufficient controllability criteria for two-player games based on fully mixed Nash equilibria and the rank of the projected payoff matrix. For multi-player settings, they propose two sufficient conditions for controllability and validate their theoretical findings through classic games such as Rock–Paper–Scissors.

Abstract

Follow-the-regularized-leader (FTRL) algorithms have become popular in the context of games, providing easy-to-implement methods for each agent, as well as theoretical guarantees that the strategies of all agents will converge to some equilibrium concept (provided that all agents follow the appropriate dynamics). However, with these methods, each agent ignores the coupling in the game, and treats their payoff vectors as exogenously given. In this paper, we take the perspective of one agent (the controller) deciding their mixed strategies in a finite game, while one or more other agents update their mixed strategies according to continuous-time FTRL. Viewing the learners' dynamics as a nonlinear control system evolving on the relative interior of a simplex or product of simplices, we ask when the controller can steer the learners to a target state, using only its own mixed strategy and without modifying the game's payoff structure. For the two-player case we provide a necessary and sufficient criterion for controllability based on the existence of a fully mixed neutralizing controller strategy and a rank condition on the projected payoff map. For multi-learner interactions we give two sufficient controllability conditions, one based on uniform neutralization and one based on a periodic-drift hypothesis together with a Lie-algebra rank condition. We illustrate these results on canonical examples such as Rock-Paper-Scissors and a construction related to Brockett's integrator.

Problem

• controllability
• FTRL
• game dynamics
• steering
• mixed strategies

Innovation

• controllability
• FTRL dynamics
• nonlinear control
• game theory
• steering learners