Coach: Exploiting Temporal Patterns for All-Resource Oversubscription in Cloud Platforms

2025-01-19

AI Summary

Cloud platforms suffer from low CPU and memory utilization, redundant resource reservations, and substantial idle resource waste. To address these issues, this paper proposes CoachVM—a holistic resource (especially memory) overcommitment system grounded in temporal complementarity patterns among virtual machines (VMs). We introduce the novel CoachVM VM type, which partitions each resource into guaranteed and overcommitted shares. Furthermore, we design the first lightweight online memory overcommitment monitoring and contention-mitigation mechanism, enabling coordinated CPU-memory overcommitment. Our approach integrates long-horizon resource demand forecasting, temporal complementarity modeling, hierarchical scheduling, and real-time memory contention detection with migration avoidance. Evaluated on real Azure production workloads, CoachVM increases VM consolidation density by 26% while limiting performance degradation to ≤1%, significantly outperforming state-of-the-art systems.

Technology Category

• Machine Learning: Hardware-aware ML
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Cloud platforms remain underutilized despite multiple proposals to improve their utilization (e.g., disaggregation, harvesting, and oversubscription). Our characterization of the resource utilization of virtual machines (VMs) in Azure reveals that, while CPU is the main underutilized resource, we need to provide a solution to manage all resources holistically. We also observe that many VMs exhibit complementary temporal patterns, which can be leveraged to improve the oversubscription of underutilized resources. Based on these insights, we propose Coach: a system that exploits temporal patterns for all-resource oversubscription in cloud platforms. Coach uses long-term predictions and an efficient VM scheduling policy to exploit temporally complementary patterns. We introduce a new general-purpose VM type, called CoachVM, where we partition each resource allocation into a guaranteed and an oversubscribed portion. Coach monitors the oversubscribed resources to detect contention and mitigate any potential performance degradation. We focus on memory management, which is particularly challenging due to memory's sensitivity to contention and the overhead required to reassign it between CoachVMs. Our experiments show that Coach enables platforms to host up to ~26% more VMs with minimal performance degradation.

Problem

• Cloud Platform Efficiency
• CPU Utilization
• Resource Optimization

Innovation

• Cloud Platform Resource Optimization
• Predictive Intelligent Scheduling
• Enhanced Virtual Machine Capacity