🤖 AI Summary
This work proposes a collaborative multi-agent reinforcement learning–based scheduling framework to address the limitations of existing Kubernetes schedulers in large-scale heterogeneous clusters, particularly regarding scalability, dynamic multi-objective optimization, and stress awareness. The approach models each node as an agent and employs a centralized training with decentralized execution paradigm, leveraging graph neural networks to construct a decentralized yet globally informed state representation. A stress-aware lexicographic policy is designed to dynamically prioritize and balance competing objectives. Experimental evaluation on Google Kubernetes Engine demonstrates that the proposed scheduler significantly outperforms the default Kubernetes scheduler, achieving superior resource utilization, fault tolerance, and cost efficiency—especially under mixed workloads comprising both batch and latency-sensitive critical tasks.
📝 Abstract
State-of-the-art cloud-native applications require intelligent schedulers that can effectively balance system stability, resource utilisation, and associated costs. While Kubernetes provides feasibility-based placement by default, recent research efforts have explored the use of reinforcement learning (RL) for more intelligent scheduling decisions. However, current RL-based schedulers have three major limitations. First, most of these schedulers use monolithic centralised agents, which are non-scalable for large heterogeneous clusters. Second, the ones that use multi-objective reward functions assume simple, static, linear combinations of the objectives. Third, no previous work has produced a stress-aware scheduler that can react adaptively to dynamic conditions. To address these gaps in current research, we propose the Adaptive Graph-enhanced Multi-Agent Reinforcement Learning Dynamic Kubernetes Scheduler (AGMARL-DKS). AGMARL-DKS addresses these gaps by introducing three major innovations. First, we construct a scalable solution by treating the scheduling challenge as a cooperative multi-agent problem, where every cluster node operates as an agent, employing centralised training methods before decentralised execution. Second, to be context-aware and yet decentralised, we use a Graph Neural Network (GNN) to build a state representation of the global cluster context at each agent. This represents an improvement over methods that rely solely on local observations. Finally, to make trade-offs between these objectives, we use a stress-aware lexicographical ordering policy instead of a simple, static linear weighting of these objectives. The evaluations in Google Kubernetes Engine (GKE) reveal that AGMARL-DKS significantly outperforms the default scheduler in terms of fault tolerance, utilisation, and cost, especially in scheduling batch and mission-critical workloads.