Network Topology and Information Efficiency of Multi-Agent Systems: Study based on MARL

2025-10-09

AI Summary

To address inefficient coordination in multi-agent reinforcement learning (MARL) caused by non-stationarity and partial observability, this work investigates two underexplored yet critical dimensions: communication topology design and information efficiency. We propose a directed-sequential hybrid communication topology that models dynamic, asynchronous, and role-aware information flow among agents. Further, we introduce two novel metrics—the Information Entropy Efficiency Index (IEI) and the Specialization Efficiency Index (SEI)—to explicitly quantify message compactness and functional differentiation, respectively, and incorporate them into the MARL training objective. Evaluated across diverse homogeneous and heterogeneous cooperative tasks, our approach achieves significant improvements: +12.7% higher task success rate, 2.3× faster convergence, and a 41.5% reduction in average communication overhead. This work establishes a new paradigm for designing efficient and interpretable inter-agent communication in MARL.

Technology Category

• Multiagent Systems: Agent Communication
• Machine Learning: Efficient ML / Green AI
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Multi-agent systems (MAS) solve complex problems through coordinated autonomous entities with individual decision-making capabilities. While Multi-Agent Reinforcement Learning (MARL) enables these agents to learn intelligent strategies, it faces challenges of non-stationarity and partial observability. Communications among agents offer a solution, but questions remain about its optimal structure and evaluation. This paper explores two underexamined aspects: communication topology and information efficiency. We demonstrate that directed and sequential topologies improve performance while reducing communication overhead across both homogeneous and heterogeneous tasks. Additionally, we introduce two metrics -- Information Entropy Efficiency Index (IEI) and Specialization Efficiency Index (SEI) -- to evaluate message compactness and role differentiation. Incorporating these metrics into training objectives improves success rates and convergence speed. Our findings highlight that designing adaptive communication topologies with information-efficient messaging is essential for effective coordination in complex MAS.

Problem

• Optimizing communication topology structure for multi-agent reinforcement learning systems
• Evaluating information efficiency in agent messaging through novel metrics
• Reducing communication overhead while improving coordination performance in MAS

Innovation

• Directed sequential topologies reduce communication overhead
• Information entropy efficiency index evaluates message compactness
• Specialization efficiency index measures role differentiation improvement