Can We Optimize the Performance-Carbon Emission Break-Even Point?: The Quest for Greener LLMs

2026-08-09

AI Summary

This work addresses the significant yet often overlooked carbon emissions incurred during the inference phase of large language models, which far exceed those from fine-tuning. To directly optimize inference carbon efficiency, the authors propose a carbon-aware fine-tuning mechanism that incorporates a lightweight, hardware-calibrated linear energy proxy—derived from empirical measurements and encompassing parameter norms, FLOPs, and memory usage—as a carbon regularization term within a joint loss function. Experiments across MMLU multi-subject benchmarks on Gemma-2, Llama-3.1, and Qwen-2.5 demonstrate that this approach improves accuracy without increasing carbon emissions, achieving zero-carbon-cost performance gains for specific model–task configurations. The findings further reveal a non-empty, architecture- and task-dependent Pareto frontier between carbon emissions and model performance.

Abstract

The carbon footprint of any deployed Large Language Model (LLM) accumulates during inference, where repeated use of the model substantially exceeds the one-time cost of fine-tuning. Yet most efficiency interventions target either pre-training scale or post-hoc compression. We ask whether folding a calibrated, differentiable energy surrogate into the fine-tuning objective can produce inference behavior that gains task accuracy at zero or near-zero carbon cost, a break-even configuration. We propose a joint loss mechanism with a per-model carbon-emission parameter, a linear surrogate over parameter norm, FLOP proxy, and a memory proxy, fit from on-hardware energy profiling. We fine-tune three architecturally distinct families: Gemma-2 2B, Llama-3.1 8B, and Qwen-2.5 14B, and evaluate inference F1 and CO$_2$ emissions on three MMLU subjects: abstract algebra, philosophy, and formal logic. We discover from several outcomes that the carbon term behaves as either harmful interference or beneficial regularization depending on the task structure. We position calibrated carbon-aware fine-tuning as a lightweight, drop-in regularizer with a non-empty but model and task-dependent break-even region. This is an ongoing work, and we will release our codebase soon.

Problem

• carbon emission
• large language models
• inference efficiency
• performance-carbon trade-off
• green AI

Innovation

• carbon-aware fine-tuning
• energy surrogate
• break-even point
• differentiable carbon proxy
• LLM efficiency