Topology-Aware Graph Reinforcement Learning for Dynamic Routing in Cloud Networks

2025-09-05

AI Summary

To address unstable routing policy decisions and insufficient structural awareness under dynamic cloud network topologies, this paper proposes a topology-aware graph reinforcement learning method. Our approach introduces two key innovations: (1) a structure-aware state encoding module that jointly leverages multi-layer graph convolutional networks and structural position encoding to explicitly capture high-order node dependencies; and (2) a policy-adaptive graph updating mechanism that dynamically reconstructs the graph topology based on routing decisions, thereby enhancing sensitivity to and adaptability against topological changes. Evaluated on the real-world GEANT topology dataset, our method significantly outperforms mainstream baselines across critical metrics—including throughput, end-to-end latency, and link load balancing—demonstrating both effectiveness and robustness in dynamic cloud networking environments.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Planning, Routing, and Scheduling: Planning/Scheduling and Learning
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

This paper proposes a topology-aware graph reinforcement learning approach to address the routing policy optimization problem in cloud server environments. The method builds a unified framework for state representation and structural evolution by integrating a Structure-Aware State Encoding (SASE) module and a Policy-Adaptive Graph Update (PAGU) mechanism. It aims to tackle the challenges of decision instability and insufficient structural awareness under dynamic topologies. The SASE module models node states through multi-layer graph convolution and structural positional embeddings, capturing high-order dependencies in the communication topology and enhancing the expressiveness of state representations. The PAGU module adjusts the graph structure based on policy behavior shifts and reward feedback, enabling adaptive structural updates in dynamic environments. Experiments are conducted on the real-world GEANT topology dataset, where the model is systematically evaluated against several representative baselines in terms of throughput, latency control, and link balance. Additional experiments, including hyperparameter sensitivity, graph sparsity perturbation, and node feature dimensionality variation, further explore the impact of structure modeling and graph updates on model stability and decision quality. Results show that the proposed method outperforms existing graph reinforcement learning models across multiple performance metrics, achieving efficient and robust routing in dynamic and complex cloud networks.

Problem

• Optimizing routing policies in dynamic cloud server environments
• Addressing decision instability under changing network topologies
• Enhancing structural awareness for adaptive routing in complex networks

Innovation

• Structure-Aware State Encoding with graph convolution
• Policy-Adaptive Graph Update mechanism for dynamics
• Topology-aware reinforcement learning for cloud routing