How Hungry is AI? Benchmarking Energy, Water, and Carbon Footprint of LLM Inference

2025-05-14

AI Summary

This work addresses three critical gaps in prior LLM sustainability research: neglect of proprietary models, infrastructure heterogeneity, and environmental impact during inference. We propose the first infrastructure-aware benchmarking framework for quantifying the environmental footprint of LLM inference. Our framework integrates empirically measured API latency and energy consumption, regional grid carbon intensity coefficients, and hardware configuration statistics inferred via statistical inversion, establishing a multidimensional evaluation paradigm. To jointly optimize performance and sustainability, we introduce a cross-efficiency Data Envelopment Analysis (DEA) model for holistic ranking. Empirical evaluation across 30 commercial LLMs deployed in real-world data centers reveals that o3 and DeepSeek-R1 consume over 33 Wh per long-prompt inference—more than 70× that of GPT-4.1 nano—while Claude-3.7 Sonnet achieves the highest eco-efficiency. Furthermore, daily processing of 700 million GPT-4o queries consumes annual electricity equivalent to that of 35,000 U.S. households.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

As large language models (LLMs) spread across industries, understanding their environmental footprint at the inference level is no longer optional; it is essential. However, most existing studies exclude proprietary models, overlook infrastructural variability and overhead, or focus solely on training, even as inference increasingly dominates AI's environmental impact. To bridge this gap, this paper introduces a novel infrastructure-aware benchmarking framework for quantifying the environmental footprint of LLM inference across 30 state-of-the-art models as deployed in commercial data centers. Our framework combines public API performance data with region-specific environmental multipliers and statistical inference of hardware configurations. We additionally utilize cross-efficiency Data Envelopment Analysis (DEA) to rank models by performance relative to environmental cost. Our results show that o3 and DeepSeek-R1 emerge as the most energy-intensive models, consuming over 33 Wh per long prompt, more than 70 times the consumption of GPT-4.1 nano, and that Claude-3.7 Sonnet ranks highest in eco-efficiency. While a single short GPT-4o query consumes 0.43 Wh, scaling this to 700 million queries/day results in substantial annual environmental impacts. These include electricity use comparable to 35,000 U.S. homes, freshwater evaporation matching the annual drinking needs of 1.2 million people, and carbon emissions requiring a Chicago-sized forest to offset. These findings illustrate a growing paradox: although individual queries are efficient, their global scale drives disproportionate resource consumption. Our study provides a standardized, empirically grounded methodology for benchmarking the sustainability of LLM deployments, laying a foundation for future environmental accountability in AI development and sustainability standards.

Problem

• Quantifying environmental footprint of LLM inference across 30 models
• Addressing gaps in existing studies on AI's energy and resource consumption
• Providing standardized benchmarking for sustainability in AI deployments

Innovation

• Infrastructure-aware benchmarking framework for LLM environmental footprint
• Combines API data, environmental multipliers, hardware inference
• Uses cross-efficiency DEA to rank models by eco-efficiency