Learning Efficient Communication Protocols for Multi-Agent Reinforcement Learning

2025-11-12

AI Summary

In multi-agent reinforcement learning (MARL), the absence of formal modeling and optimization of communication protocols often leads to information redundancy and suboptimal communication efficiency. Method: This paper proposes a communication-efficiency-oriented joint optimization framework. First, it introduces three differentiable communication efficiency metrics—information entropy efficiency index, specialization efficiency index, and topology efficiency index—to quantitatively evaluate communication quality. Second, it designs a loss function incorporating information entropy constraints and role-specialization regularization to jointly optimize both communication topology and message generation policies within a multi-round communication paradigm. Results: Experiments across multiple cooperative tasks demonstrate that the proposed method significantly reduces communication overhead while improving task success rates and collaboration robustness. It establishes a novel paradigm for learning efficient and interpretable communication protocols in MARL.

Technology Category

• Multiagent Systems: Agent Communication
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Multi-Agent Systems (MAS) have emerged as a powerful paradigm for modeling complex interactions among autonomous entities in distributed environments. In Multi-Agent Reinforcement Learning (MARL), communication enables coordination but can lead to inefficient information exchange, since agents may generate redundant or non-essential messages. While prior work has focused on boosting task performance with information exchange, the existing research lacks a thorough investigation of both the appropriate definition and the optimization of communication protocols (communication topology and message). To fill this gap, we introduce a generalized framework for learning multi-round communication protocols that are both effective and efficient. Within this framework, we propose three novel Communication Efficiency Metrics (CEMs) to guide and evaluate the learning process: the Information Entropy Efficiency Index (IEI) and Specialization Efficiency Index (SEI) for efficiency-augmented optimization, and the Topology Efficiency Index (TEI) for explicit evaluation. We integrate IEI and SEI as the adjusted loss functions to promote informative messaging and role specialization, while using TEI to quantify the trade-off between communication volume and task performance. Through comprehensive experiments, we demonstrate that our learned communication protocol can significantly enhance communication efficiency and achieves better cooperation performance with improved success rates.

Problem

• Optimizing communication protocols to reduce redundant messages in multi-agent reinforcement learning
• Developing efficiency metrics to evaluate communication topology and message quality
• Enhancing coordination performance while minimizing communication volume in distributed systems

Innovation

• Learning multi-round communication protocols for efficiency
• Introducing three Communication Efficiency Metrics for optimization
• Integrating adjusted loss functions for informative messaging