DRAMA: A Dynamic Packet Routing Algorithm using Multi-Agent Reinforcement Learning with Emergent Communication

2025-04-06

AI Summary

Existing RL and MARL routing algorithms exhibit poor adaptability to dynamic networks, struggling with real-time topology changes and incurring high communication overhead. To address these limitations, this paper proposes a multi-agent reinforcement learning–based dynamic packet routing algorithm. Its core contributions are: (1) a graph-structure-driven emergent communication mechanism enabling runtime self-organized coordination among routers; and (2) a scalable, graph neural network–enhanced distributed Q-learning architecture supporting zero-shot topology generalization and plug-and-play deployment. Crucially, the method adapts to previously unseen network topologies without retraining. Evaluated under dynamic traffic loads and frequent topology changes, it achieves a 12.7% improvement in packet delivery ratio and a 19.3% reduction in end-to-end latency compared to state-of-the-art baselines, demonstrating superior robustness and scalability.

Technology Category

• Multiagent Systems: Multiagent Learning
• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The continuous expansion of network data presents a pressing challenge for conventional routing algorithms. As the demand escalates, these algorithms are struggling to cope. In this context, reinforcement learning (RL) and multi-agent reinforcement learning (MARL) algorithms emerge as promising solutions. However, the urgency and importance of the problem are clear, as existing RL/MARL-based routing approaches lack effective communication in run time among routers, making it challenging for individual routers to adapt to complex and dynamic changing networks. More importantly, they lack the ability to deal with dynamically changing network topology, especially the addition of the router, due to the non-scalability of their neural networks. This paper proposes a novel dynamic routing algorithm, DRAMA, incorporating emergent communication in multi-agent reinforcement learning. Through emergent communication, routers could learn how to communicate effectively to maximize the optimization objectives. Meanwhile, a new Q-network and graph-based emergent communication are introduced to dynamically adapt to the changing network topology without retraining while ensuring robust performance. Experimental results showcase DRAMA's superior performance over the traditional routing algorithm and other RL/MARL-based algorithms, achieving a higher delivery rate and lower latency in diverse network scenarios, including dynamic network load and topology. Moreover, an ablation experiment validates the prospect of emergent communication in facilitating packet routing.

Problem

• Dynamic packet routing in complex networks using MARL
• Enabling effective real-time communication among routers
• Adapting to dynamically changing network topologies

Innovation

• Multi-agent reinforcement learning with emergent communication
• Dynamic Q-network for changing network topology
• Graph-based communication without retraining