Optimising for Energy Efficiency and Performance in Machine Learning

2026-01-13

AI Summary

This work addresses the lack of effective evaluation and optimization methods for energy consumption and environmental impact during machine learning inference, as well as the limited actionable feedback provided by existing tools. To this end, we propose ECOpt, an energy consumption optimizer that, for the first time, jointly models model performance and energy efficiency. ECOpt leverages automated hyperparameter tuning and hardware-agnostic energy measurement to generate an interpretable Pareto frontier that quantifies the trade-off between accuracy and efficiency. Experiments reveal that parameter count and FLOPs are unreliable proxies for energy usage, that Transformer-based text generation models exhibit consistent energy efficiency across diverse hardware, and that seven state-of-the-art models on CIFAR-10 achieve optimal balance between accuracy and energy efficiency. ECOpt not only facilitates green deployment decisions but also demonstrates net-positive environmental benefits, advancing the standardization of energy-efficiency metrics for ML models.

Technology Category

• Machine Learning: Efficient ML / Green AI
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Security and Privacy: Security and privacy of machine learning and AI applications

Abstract

The ubiquity of machine learning (ML) and the demand for ever-larger models bring an increase in energy consumption and environmental impact. However, little is known about the energy scaling laws in ML, and existing research focuses on training cost -- ignoring the larger cost of inference. Furthermore, tools for measuring the energy consumption of ML do not provide actionable feedback. To address these gaps, we developed Energy Consumption Optimiser (ECOpt): a hyperparameter tuner that optimises for energy efficiency and model performance. ECOpt quantifies the trade-off between these metrics as an interpretable Pareto frontier. This enables ML practitioners to make informed decisions about energy cost and environmental impact, while maximising the benefit of their models and complying with new regulations. Using ECOpt, we show that parameter and floating-point operation counts can be unreliable proxies for energy consumption, and observe that the energy efficiency of Transformer models for text generation is relatively consistent across hardware. These findings motivate measuring and publishing the energy metrics of ML models. We further show that ECOpt can have a net positive environmental impact and use it to uncover seven models for CIFAR-10 that improve upon the state of the art, when considering accuracy and energy efficiency together.

Problem

• energy efficiency
• machine learning
• inference cost
• environmental impact
• energy consumption

Innovation

• energy efficiency
• hyperparameter optimization
• Pareto frontier
• inference cost
• sustainable machine learning