An Overlay Multicast Routing Method Based on Network Situational Aware-ness and Hierarchical Multi-Agent Reinforcement Learning

2026-01-17

Electronic Research Archive

AI Summary

This work addresses the limitations of traditional overlay multicast in adapting to dynamic traffic fluctuations and the high complexity, slow convergence, and instability of existing reinforcement learning approaches, which often fail to effectively decouple multi-objective optimization. Leveraging the global network view provided by Software-Defined Networking (SDN), the authors propose a hierarchical multi-agent deep reinforcement learning architecture that decomposes multicast tree construction into two coordinated stages. This design significantly reduces the action space and disentangles competing optimization objectives. Experimental results demonstrate that the proposed method outperforms state-of-the-art solutions in terms of end-to-end delay, bandwidth utilization, and packet loss rate, while also achieving faster convergence, superior scalability, and enhanced routing adaptability and flexibility.

Technology Category

• Multiagent Systems: Multiagent Learning
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Compared with IP multicast, Overlay Multicast (OM) offers better compatibility and flexible deployment in heterogeneous, cross-domain networks. However, traditional OM struggles to adapt to dynamic traffic due to unawareness of physical resource states, and existing reinforcement learning methods fail to decouple OM's tightly coupled multi-objective nature, leading to high complexity, slow convergence, and instability. To address this, we propose MA-DHRL-OM, a multi-agent deep hierarchical reinforcement learning approach. Using SDN's global view, it builds a traffic-aware model for OM path planning. The method decomposes OM tree construction into two stages via hierarchical agents, reducing action space and improving convergence stability. Multi-agent collaboration balances multi-objective optimization while enhancing scalability and adaptability. Experiments show MA-DHRL-OM outperforms existing methods in delay, bandwidth utilization, and packet loss, with more stable convergence and flexible routing.

Problem

• Overlay Multicast
• Network Situational Awareness
• Multi-Agent Reinforcement Learning
• Multi-Objective Optimization
• Dynamic Traffic Adaptation

Innovation

• Overlay Multicast
• Hierarchical Multi-Agent Reinforcement Learning
• Network Situational Awareness
• SDN-based Traffic Awareness
• Multi-Objective Optimization