Towards Sustainable Large Language Model Serving

2024-12-31

AI Summary

Prior studies on LLM carbon footprint overlook embodied emissions (e.g., chip manufacturing) and regional grid heterogeneity, leading to incomplete sustainability assessments. Method: We propose the first holistic sustainability evaluation framework jointly modeling operational carbon emissions (during inference) and embodied emissions (from GPU fabrication), enabling cross-hardware (RTX6000 Ada/T4) and cross-regional (varying grid carbon intensities) analysis. Using empirical measurements of LLaMA-1B/3B/7B inference power consumption, die area, and memory parameters, we quantify embodied emissions and integrate region-specific grid carbon intensities for end-to-end emission accounting. Contribution/Results: Experiments reveal strong coupling among hardware generation, model scale, and grid cleanliness in determining total carbon footprint. We open-source a reproducible assessment toolkit, providing theoretical foundations and actionable optimization pathways for low-carbon LLM deployment.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

In this work, we study LLMs from a carbon emission perspective, addressing both operational and embodied emissions, and paving the way for sustainable LLM serving. We characterize the performance and energy of LLaMA with 1B, 3B, and 7B parameters using two Nvidia GPU types, a latest-generation RTX6000 Ada and an older-generation T4. We analytically model operational carbon emissions based on energy consumption and carbon intensities from three grid regions -- each representing a different energy source mix, and embodied carbon emissions based on chip area and memory size. Our characterization and modeling provide us with an in-depth understanding of the performance, energy, and carbon emissions of LLM serving. Our findings highlight the potential for optimizing sustainable LLM serving systems by considering both operational and embodied carbon emissions simultaneously.

Problem

• Carbon Emissions
• Large Language Models
• Environmental Sustainability

Innovation

• Carbon Emission Assessment
• LLMs Energy Efficiency
• Sustainable Computing