Towards Resource-Efficient LLMs: End-to-End Energy Accounting of Distillation Pipelines

2026-05-13

AI Summary

This work addresses the common oversight in existing large language model (LLM) distillation methods of neglecting end-to-end energy consumption on the teacher model side—encompassing data generation, logit caching, and evaluation. The study proposes the first systematic framework for accounting energy costs, employing fine-grained GPU power monitoring to empirically measure energy use and carbon emissions across the entire distillation pipeline. It quantifies previously hidden energy costs per stage, constructs an energy–quality Pareto frontier, and introduces distillation strategies and hyperparameter selection criteria tailored to energy-efficiency constraints. To support reproducible green AI research, the authors open-source a standardized measurement protocol and toolchain, revealing high-cost components overlooked by conventional approaches and establishing a benchmark for sustainable LLM development.

Abstract

The rise in deployment of large language models has driven a surge in GPU demand and datacenter scaling, raising concerns about electricity use, grid stress, and the impacts of modern AI workloads. Distillation is often promoted as one of the most effective paths to obtain cheaper, more efficient models, yet these claims rarely account for the full end-to-end energy and resource costs, including crucial teacher-side workloads such as data generation, logit caching, and evaluation. We present a comprehensive energy accounting framework that measures the complete computational cost of distillation pipelines via detailed stage-wise tracking of GPU device power consumption. In our experiments, we separate and log empirical energy use across distinct phases and systematically measure the energy and emissions of two common distillation methods: the classic logit-based knowledge distillation and synthetic-data supervised fine-tuning, constructing energy-quality Pareto frontiers that expose the previously ignored costs. From these measurements and analyses, we derive practical design rules for selecting distillation methods and hyperparameters under energy and budget constraints, and release an open-source measurement harness and accounting protocol to provide a standardized foundation for comparable, reproducible distillation research, explicitly accountable for complete pipeline energy impact.

Problem

• energy accounting
• large language models
• knowledge distillation
• resource efficiency
• carbon emissions

Innovation

• energy accounting
• knowledge distillation
• resource-efficient LLMs
• GPU power consumption
• Pareto frontier