Aging-aware CPU Core Management for Embodied Carbon Amortization in Cloud LLM Inference

2025-01-27

AI Summary

In cloud-based large language model (LLM) inference, rapid accumulation of CPU embodied carbon and accelerated silicon aging lead to premature hardware retirement. Method: This paper proposes an aging-aware CPU core dynamic management mechanism that jointly integrates selective deep-idle scheduling with aging-balanced core allocation. It proactively mitigates silicon degradation while bounding P99 latency increase to under 10%. Contribution/Results: Evaluated on real Azure inference traces and an extended Microsoft LLM cluster simulator—with fine-grained aging modeling and dynamic core scheduling—the approach reduces annual embodied carbon by 37.67% and improves CPU utilization by 77%. To our knowledge, this is the first work to tightly couple physics-informed silicon aging models with green AI inference scheduling, delivering a deployable, system-level solution for low-carbon cloud infrastructure.

Technology Category

• Machine Learning: Hardware-aware ML
• Planning, Routing, and Scheduling: Planning with Language Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Broad adoption of Large Language Models (LLM) demands rapid expansions of cloud LLM inference clusters, leading to accumulation of embodied carbon$-$the emissions from manufacturing and supplying IT assets$-$that mostly concentrate on inference server CPU. This paper delves into the challenges of sustainable growth of cloud LLM inference, emphasizing extended amortization of CPU embodied over an increased lifespan. Given the reliability risks of silicon aging, we propose an aging-aware CPU core management technique to delay CPU aging effects, allowing the cluster operator to safely increase CPU life. Our technique exploits CPU underutilization patterns that we uncover in cloud LLM inference by halting aging in unused cores and even-outing aging in active cores via selective deep idling and aging-aware inference task allocation. Through extensive simulations using real-world Azure inference traces and an extended LLM cluster simulator from Microsoft, we show superior performance of our technique over existing methods with an estimated 37.67% reduction in yearly embodied carbon emissions through p99 performance of managing CPU aging effects, a 77% reduction in CPU underutilization, and less than 10% impact to the inference service quality.

Problem

• Carbon Emissions
• Energy Efficiency
• Sustainable Computing

Innovation

• Smart CPU Aging Management
• Carbon Emission Reduction
• Service Quality Preservation