Learning Virtual Machine Scheduling in Cloud Computing through Language Agents

2025-05-15

AI Summary

Virtual machine scheduling in cloud computing constitutes an online dynamic multi-dimensional bin packing (ODMBP) problem, where conventional heuristic strategies lack adaptability and learning-based approaches suffer from poor generalizability and interpretability. Method: We propose MiCo—the first large language model (LLM)-driven hierarchical language agent framework that formalizes heuristic design. Grounded in a semi-Markov decision process (SMDP), MiCo features a two-stage architecture—Option Miner and Composer—that jointly enables automatic discovery, composition, and context-aware adaptation of scheduling policies. Contribution/Results: Evaluated on a real-world enterprise dataset comprising over 10,000 VMs, MiCo achieves a competitive ratio of 96.9%, significantly outperforming state-of-the-art baselines. It demonstrates strong robustness against non-stationary request streams and heterogeneous resource configurations, while preserving policy interpretability and generalizability.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

In cloud services, virtual machine (VM) scheduling is a typical Online Dynamic Multidimensional Bin Packing (ODMBP) problem, characterized by large-scale complexity and fluctuating demands. Traditional optimization methods struggle to adapt to real-time changes, domain-expert-designed heuristic approaches suffer from rigid strategies, and existing learning-based methods often lack generalizability and interpretability. To address these limitations, this paper proposes a hierarchical language agent framework named MiCo, which provides a large language model (LLM)-driven heuristic design paradigm for solving ODMBP. Specifically, ODMBP is formulated as a Semi-Markov Decision Process with Options (SMDP-Option), enabling dynamic scheduling through a two-stage architecture, i.e., Option Miner and Option Composer. Option Miner utilizes LLMs to discover diverse and useful non-context-aware strategies by interacting with constructed environments. Option Composer employs LLMs to discover a composing strategy that integrates the non-context-aware strategies with the contextual ones. Extensive experiments on real-world enterprise datasets demonstrate that MiCo achieves a 96.9% competitive ratio in large-scale scenarios involving more than 10,000 virtual machines. It maintains high performance even under nonstationary request flows and diverse configurations, thus validating its effectiveness in complex and large-scale cloud environments.

Problem

• Addressing real-time adaptation in VM scheduling for cloud services
• Overcoming rigidity in heuristic-based VM scheduling approaches
• Enhancing generalizability and interpretability in learning-based VM scheduling

Innovation

• Hierarchical language agent framework for VM scheduling
• LLM-driven heuristic design for ODMBP problem
• Two-stage SMDP-Option architecture for dynamic scheduling