Hybrid Learning for Cold-Start-Aware Microservice Scheduling in Dynamic Edge Environments

2025-05-28

AI Summary

Dynamic edge environments pose dual challenges for microservice scheduling: time-varying resource availability and low efficiency during reinforcement learning (RL) cold-start phases. Method: This paper proposes a hybrid learning framework integrating offline imitation learning with online Soft Actor-Critic (SAC). It introduces a novel cold-start-aware learning paradigm; designs a GRU-enhanced policy network to decouple slowly varying node states from rapidly changing service features; incorporates an action-selection mechanism to accelerate convergence; and integrates a rule-based expert system for joint latency–energy optimization. Contribution/Results: Experiments demonstrate a 50% improvement in the composite objective function and a 70% reduction in convergence time compared to baseline RL and heuristic methods. The framework significantly enhances scheduling stability and robustness across diverse edge configurations, including heterogeneous hardware, dynamic workloads, and volatile network conditions.

Technology Category

• Planning, Routing, and Scheduling: Planning/Scheduling and Learning
• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

With the rapid growth of IoT devices and their diverse workloads, container-based microservices deployed at edge nodes have become a lightweight and scalable solution. However, existing microservice scheduling algorithms often assume static resource availability, which is unrealistic when multiple containers are assigned to an edge node. Besides, containers suffer from cold-start inefficiencies during early-stage training in currently popular reinforcement learning (RL) algorithms. In this paper, we propose a hybrid learning framework that combines offline imitation learning (IL) with online Soft Actor-Critic (SAC) optimization to enable a cold-start-aware microservice scheduling with dynamic allocation for computing resources. We first formulate a delay-and-energy-aware scheduling problem and construct a rule-based expert to generate demonstration data for behavior cloning. Then, a GRU-enhanced policy network is designed in the policy network to extract the correlation among multiple decisions by separately encoding slow-evolving node states and fast-changing microservice features, and an action selection mechanism is given to speed up the convergence. Extensive experiments show that our method significantly accelerates convergence and achieves superior final performance. Compared with baselines, our algorithm improves the total objective by $50%$ and convergence speed by $70%$, and demonstrates the highest stability and robustness across various edge configurations.

Problem

• Dynamic resource allocation for microservices in edge environments
• Cold-start inefficiencies in reinforcement learning for scheduling
• Delay-and-energy-aware microservice scheduling optimization

Innovation

• Hybrid learning combines IL and SAC optimization
• GRU-enhanced policy network encodes node states
• Action selection mechanism speeds up convergence