Understanding the Energy Scaling of Large Language Model Inference Across Context Lengths and Attention Architectures

2026-08-25

AI Summary

本文通过研究不同注意力机制和上下文长度下的能量消耗，解决了大型语言模型推理过程中的能耗问题，提供了选择节能架构的指导。

Abstract

The growing adoption of large language models (LLMs) has raised increasing concerns about the energy consumption and environmental impact of inference. This paper presents a systematic empirical study of decode-phase energy consumption across representative open-source LLMs employing Multi-Head Attention (MHA), Grouped Query Attention (GQA), and Grouped Query Attention with Sliding Window Attention (SWA) to characterize how attention architecture influences decode-phase energy consumption under varying inference workloads. We evaluate four models across different context lengths, batch sizes, and generation workloads while measuring GPU energy using NVIDIA hardware counters. We examine the effects of context length, attention mechanism, Key-Value (KV) cache growth, and batching on decode-phase energy consumption. Results show that attention mechanism is the primary factor governing how decode energy scales with context length. MHA models exhibit substantially steeper energy growth than GQA models, whereas GQA with SWA maintains nearly constant energy consumption. We further show that model size primarily determines absolute energy consumption, while batching reduces both energy per generated token and request latency by up to 87%. These findings provide practical guidance for selecting energy-efficient LLM architectures and inference configurations.

Problem

• energy consumption
• large language models
• inference
• attention architecture
• context length

Innovation

• attention mechanism
• energy consumption
• context length
• batching