Multi-Agent Online Control with Adversarial Disturbances

2025-06-23

AI Summary

This paper studies online control of large-scale multi-agent systems under adversarial perturbations: agents are numerous, objectives are time-varying and competitive, and no coordination assumptions are imposed. We propose a distributed gradient-based controller that achieves near-optimal sublinear individual regret—specifically, $O(sqrt{T})$—under minimal communication requirements. Notably, we derive the first explicit characterization of how the regret bound scales with the number of agents $n$. For the special case of common objectives, we formulate a time-varying potential game model and prove bounded equilibrium gap and convergence. Our approach integrates online convex optimization, robust control theory, and potential game analysis, thereby overcoming conventional reliance on either cooperative behavior or stochastic disturbance assumptions. Experiments demonstrate that all agents consistently achieve strong robust individual performance. This work establishes both theoretical foundations and algorithmic guarantees for non-cooperative multi-agent online control.

Technology Category

• Multiagent Systems: Adversarial Agents
• Game Theory and Economic Paradigms: Adversarial Learning
• Machine Learning: Online Learning & Bandits

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Multi-agent control problems involving a large number of agents with competing and time-varying objectives are increasingly prevalent in applications across robotics, economics, and energy systems. In this paper, we study online control in multi-agent linear dynamical systems with disturbances. In contrast to most prior work in multi-agent control, we consider an online setting where disturbances are adversarial and where each agent seeks to minimize its own, adversarial sequence of convex losses. In this setting, we investigate the robustness of gradient-based controllers from single-agent online control, with a particular focus on understanding how individual regret guarantees are influenced by the number of agents in the system. Under minimal communication assumptions, we prove near-optimal sublinear regret bounds that hold uniformly for all agents. Finally, when the objectives of the agents are aligned, we show that the multi-agent control problem induces a time-varying potential game for which we derive equilibrium gap guarantees.

Problem

• Online control in multi-agent systems with adversarial disturbances
• Robustness of gradient-based controllers in adversarial settings
• Regret bounds and equilibrium guarantees for competing agents

Innovation

• Gradient-based controllers for adversarial disturbances
• Near-optimal sublinear regret bounds
• Time-varying potential game for aligned objectives