Unveiling Environmental Impacts of Large Language Model Serving: A Functional Unit View

2025-02-16

AI Summary

The absence of standardized benchmarks for quantifying carbon emissions in large language model (LLM) inference services hinders fair, cross-model, cross-configuration, and cross-hardware sustainability evaluation. Method: This paper introduces FUEL, the first Functional Unit (FU)-driven carbon impact assessment framework. FUEL unifies environmental impact measurement by mapping it to a standardized FU—“one effective inference task”—enabling comparable carbon accounting across models, configurations, and hardware platforms. It integrates FU-based modeling, cradle-to-gate carbon footprint analysis, and multi-dimensional empirical benchmarking (latency, energy consumption, CO₂-equivalent emissions). Contribution/Results: FUEL establishes a standardized FU-based metric paradigm; empirically uncovers sustainability trade-offs among model scale, quantization precision, and hardware selection; and demonstrates that synergistic optimization—including model lightweighting, operator-level acceleration, and hardware-aware deployment—reduces service carbon emissions by 30–65%. FUEL provides a reusable evaluation standard and systematic optimization methodology for green AI.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Sustainability and carbon-aware systems for Web, mobile, and WoT
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large language models (LLMs) offer powerful capabilities but come with significant environmental costs, particularly in carbon emissions. Existing studies benchmark these emissions but lack a standardized basis for comparison across models. To address this, we introduce the concept of a functional unit (FU) and develop FUEL, the first FU-based framework for evaluating LLM serving's environmental impact. Through case studies on model size, quantization, and hardware, we uncover key trade-offs in sustainability. Our findings highlight the potential for reducing carbon emissions by optimizing model selection, deployment strategies, and hardware choices, paving the way for more sustainable AI infrastructure.

Problem

• Evaluating environmental impacts of large language models
• Standardizing comparison of carbon emissions across models
• Optimizing model selection for sustainable AI infrastructure

Innovation

• Introduces functional unit concept
• Develops FUEL framework
• Optimizes model and hardware