ML-EcoLyzer: Quantifying the Environmental Cost of Machine Learning Inference Across Frameworks and Hardware
The environmental impact of machine learning inference—encompassing carbon emissions, energy consumption, thermal dissipation, and water usage—lacks systematic quantification on resource-constrained hardware. Method: We propose the first fine-grained, cross-framework (CPU, consumer-grade GPU, datacenter accelerators), multi-model, multi-task environmental impact assessment methodology, integrating adaptive runtime monitoring with hardware-aware measurement techniques. We introduce the Environmental Sustainability Score (ESS)—defined as the number of effective model parameters served per unit CO₂-equivalent emission—to expose inefficiencies of large accelerators on lightweight inference tasks. Contribution/Results: Based on empirical analysis of 1,900+ inference configurations, we establish, for the first time, a quantitative relationship between model efficiency and environmental cost. We publicly release an open-source benchmark toolkit and dataset, enabling reproducible evaluation and informed decision-making for green AI deployment.