CPU-Limits kill Performance: Time to rethink Resource Control

2025-10-12

AI Summary

This work challenges the necessity and efficacy of CPU throttling—a widely adopted resource-limiting mechanism in cloud computing. Through empirical analysis and systematic measurement in production cloud environments, the authors evaluate its impact on latency-sensitive applications, revealing that CPU limits consistently exacerbate tail latency, reduce CPU utilization, and increase per-request cost across most deployment scenarios. Methodologically, the study combines controlled microbenchmarks, production workload traces, and cross-cloud infrastructure profiling to isolate and quantify throttling-induced performance degradation. The key contributions are: (1) the first comprehensive empirical demonstration of the detrimental effects of CPU limiting; (2) a paradigm shift toward “no-limit-by-default, enable-on-demand”; (3) a reorientation of autoscaling policies from quota-based to performance-objective-driven control; and (4) empirically grounded design principles for CPU-unconstrained elastic resource management. This work critically questions long-standing industry practices and provides foundational evidence for rethinking cloud-native scheduling and pricing models.

Technology Category

• Machine Learning: Scalability of ML Systems
• Planning, Routing, and Scheduling: Scheduling under Uncertainty
• Search and Optimization: Other Foundations of Search & Optimization

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Research in compute resource management for cloud-native applications is dominated by the problem of setting optimal CPU limits -- a fundamental OS mechanism that strictly restricts a container's CPU usage to its specified CPU-limits . Rightsizing and autoscaling works have innovated on allocation/scaling policies assuming the ubiquity and necessity of CPU-limits . We question this. Practical experiences of cloud users indicate that CPU-limits harms application performance and costs more than it helps. These observations are in contradiction to the conventional wisdom presented in both academic research and industry best practices. We argue that this indiscriminate adoption of CPU-limits is driven by erroneous beliefs that CPU-limits is essential for operational and safety purposes. We provide empirical evidence making a case for eschewing CPU-limits completely from latency-sensitive applications. This prompts a fundamental rethinking of auto-scaling and billing paradigms and opens new research avenues. Finally, we highlight specific scenarios where CPU-limits can be beneficial if used in a well-reasoned way (e.g. background jobs).

Problem

• Challenging the necessity of CPU limits for performance optimization
• Demonstrating CPU limits harm latency-sensitive application performance
• Rethinking autoscaling and billing paradigms without CPU limits

Innovation

• Proposes eliminating CPU limits for latency-sensitive applications
• Challenges conventional CPU limits necessity assumptions
• Recommends selective CPU limits for background jobs