GreenLLM: Disaggregating Large Language Model Serving on Heterogeneous GPUs for Lower Carbon Emissions

2024-12-29

AI Summary

Large language model (LLM) inference services incur substantial carbon emissions due to intensive deployment of high-performance GPUs and accelerated electronic waste generation. Method: We propose the first SLO-aware heterogeneous GPU decoupling framework, which repurposes legacy, low-performance GPUs by decomposing compute workloads according to latency sensitivity. We establish a joint theoretical model integrating grid carbon intensity, GPU residual lifetime, and decoupling benefits, and jointly optimize computation offloading and load balancing via carbon-aware scheduling and delay-constrained modeling. Results: Experiments show that, while meeting SLOs for over 90% of requests, our approach reduces carbon emissions by up to 40.6% compared to an all-new GPU deployment. It significantly improves carbon efficiency over hardware lifecycles and supports multi-scenario, multi-region carbon-intensity forecasting and dynamic GPU aging modeling.

Technology Category

• Machine Learning: Hardware-aware ML
• Planning, Routing, and Scheduling: Planning with Language Models
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

LLMs have been widely adopted across many real-world applications. However, their widespread use comes with significant environmental costs due to their high computational intensity and resource demands. Specifically, this has driven the development of new generations of high-performing GPUs, exacerbating the problem of electronic waste and accelerating the premature disposal of devices. To address this problem, this paper focuses on reducing the carbon emissions of LLM serving by reusing older, low-performing GPUs. We present GreenLLM, an SLO-aware LLM serving framework designed to minimize carbon emissions by reusing older GPUs. GreenLLM builds on two identified use cases that disaggregate specific computations onto older GPUs, reducing carbon emissions while meeting performance goals. To deepen our understanding of the potential carbon savings from disaggregation, we also provide a theoretical analysis of its relationship with carbon intensity and GPU lifetime. Our evaluations show that GreenLLM reduces carbon emissions by up to 40.6% compared to running standard LLM serving on new GPU only, meeting latency SLOs for over 90% of requests across various applications, latency requirements, carbon intensities, and GPU lifetimes.

Problem

• Large Language Models
• Environmental Impact
• Electronic Waste

Innovation

• GreenLLM
• Sustainable Computing
• Pollution Reduction