🤖 AI Summary
The environmental impact of machine learning inference—encompassing carbon emissions, energy consumption, thermal dissipation, and water usage—lacks systematic quantification on resource-constrained hardware. Method: We propose the first fine-grained, cross-framework (CPU, consumer-grade GPU, datacenter accelerators), multi-model, multi-task environmental impact assessment methodology, integrating adaptive runtime monitoring with hardware-aware measurement techniques. We introduce the Environmental Sustainability Score (ESS)—defined as the number of effective model parameters served per unit CO₂-equivalent emission—to expose inefficiencies of large accelerators on lightweight inference tasks. Contribution/Results: Based on empirical analysis of 1,900+ inference configurations, we establish, for the first time, a quantitative relationship between model efficiency and environmental cost. We publicly release an open-source benchmark toolkit and dataset, enabling reproducible evaluation and informed decision-making for green AI deployment.
📝 Abstract
Machine learning inference occurs at a massive scale, yet its environmental impact remains poorly quantified, especially on low-resource hardware. We present ML-EcoLyzer, a cross-framework tool for measuring the carbon, energy, thermal, and water costs of inference across CPUs, consumer GPUs, and datacenter accelerators. The tool supports both classical and modern models, applying adaptive monitoring and hardware-aware evaluation. We introduce the Environmental Sustainability Score (ESS), which quantifies the number of effective parameters served per gram of CO$_2$ emitted. Our evaluation covers over 1,900 inference configurations, spanning diverse model architectures, task modalities (text, vision, audio, tabular), hardware types, and precision levels. These rigorous and reliable measurements demonstrate that quantization enhances ESS, huge accelerators can be inefficient for lightweight applications, and even small models may incur significant costs when implemented suboptimally. ML-EcoLyzer sets a standard for sustainability-conscious model selection and offers an extensive empirical evaluation of environmental costs during inference.