Learning Communication Skills in Multi-task Multi-agent Deep Reinforcement Learning

2025-11-05

AI Summary

To address the non-transferability of communication protocols and the difficulty of cross-task knowledge reuse in multi-task multi-agent deep reinforcement learning (MADRL), this paper proposes the Multi-task Communication Skills (MCS) framework. MCS employs a Transformer encoder to construct a task-agnostic, shared message space, enabling unified representation and cross-task transfer of communication skills; it further introduces a predictive network to explicitly model the message–action relationship, enhancing coordination between communication and policy learning. To the best of our knowledge, MCS is the first multi-task MADRL method supporting communication skill sharing and transfer. Evaluated on three multi-task benchmark environments, MCS significantly outperforms both non-communicative multi-task baselines and single-task baselines (with and without communication), achieving an average task performance improvement of 18.7% and a 32.4% gain in sample efficiency.

Technology Category

• Multiagent Systems: Agent Communication
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multi-Robot Systems

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In multi-agent deep reinforcement learning (MADRL), agents can communicate with one another to perform a task in a coordinated manner. When multiple tasks are involved, agents can also leverage knowledge from one task to improve learning in other tasks. In this paper, we propose Multi-task Communication Skills (MCS), a MADRL with communication method that learns and performs multiple tasks simultaneously, with agents interacting through learnable communication protocols. MCS employs a Transformer encoder to encode task-specific observations into a shared message space, capturing shared communication skills among agents. To enhance coordination among agents, we introduce a prediction network that correlates messages with the actions of sender agents in each task. We adapt three multi-agent benchmark environments to multi-task settings, where the number of agents as well as the observation and action spaces vary across tasks. Experimental results demonstrate that MCS achieves better performance than multi-task MADRL baselines without communication, as well as single-task MADRL baselines with and without communication.

Problem

• Learning multi-task communication protocols for multi-agent coordination
• Encoding task-specific observations into shared message space
• Enhancing agent coordination through message-action correlation prediction

Innovation

• Transformer encoder encodes observations into shared message space
• Prediction network correlates messages with sender agent actions
• Learnable communication protocols enable multi-task multi-agent coordination