Robust Multi-agent Communication Based on Decentralization-Oriented Adversarial Training

2025-04-30

AI Summary

Existing multi-agent reinforcement learning (MARL) communication strategies often converge to suboptimal equilibria, resulting in highly concentrated and structurally imbalanced channel usage—where failure of a single critical channel triggers systemic collapse. Method: We propose a decentralized adversarial training framework—the first to incorporate sociological decentralization theory into MARL communication modeling. Our Dynamic Masking Adversarial Channel module (DMAC_Adv) actively identifies and masks overloaded channels during training, steering policies toward balanced, robust, decentralized topologies. The approach integrates differentiable communication architectures with MARL, requiring no centralized coordination. Contribution/Results: Evaluated on four standard MARL benchmarks, our method significantly improves communication robustness and task performance while achieving low communication overhead and high fault tolerance. It enables scalable, resilient decentralized communication topologies without structural fragility or central dependency.

Technology Category

• Multiagent Systems: Adversarial Agents
• Machine Learning: Adversarial Learning & Robustness
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Systems and Infrastructure for Web, Mobile and WoT: Decentralized Web and Fediverse systems

Abstract

In typical multi-agent reinforcement learning (MARL) problems, communication is important for agents to share information and make the right decisions. However, due to the complexity of training multi-agent communication, existing methods often fall into the dilemma of local optimization, which leads to the concentration of communication in a limited number of channels and presents an unbalanced structure. Such unbalanced communication policy are vulnerable to abnormal conditions, where the damage of critical communication channels can trigger the crash of the entire system. Inspired by decentralization theory in sociology, we propose DMAC, which enhances the robustness of multi-agent communication policies by retraining them into decentralized patterns. Specifically, we train an adversary DMAC_Adv which can dynamically identify and mask the critical communication channels, and then apply the adversarial samples generated by DMAC_Adv to the adversarial learning of the communication policy to force the policy in exploring other potential communication schemes and transition to a decentralized structure. As a training method to improve robustness, DMAC can be fused with any learnable communication policy algorithm. The experimental results in two communication policies and four multi-agent tasks demonstrate that DMAC achieves higher improvement on robustness and performance of communication policy compared with two state-of-the-art and commonly-used baselines. Also, the results demonstrate that DMAC can achieve decentralized communication structure with acceptable communication cost.

Problem

• Enhances robustness in multi-agent communication policies
• Prevents over-reliance on limited critical communication channels
• Promotes decentralized communication structures for system resilience

Innovation

• Decentralization-oriented adversarial training for robustness
• Dynamic critical channel identification and masking
• Fusion with any learnable communication policy algorithm