WattLayer: Get Layers Right to Estimate Inference Energy of Neural Networks

2026-06-26

AI Summary

This work addresses the lack of standardized and accurate estimation of neural network inference energy consumption across diverse tasks and architectures. The authors propose a task-agnostic, layer-wise energy modeling approach that leverages hardware-aware features and a shared-layer transfer mechanism to generalize to new tasks and architectures without retraining. Built upon a large-scale dataset encompassing over 100,000 layers, the resulting regression model enables, for the first time, universal energy prediction across both tasks and hardware platforms. Evaluation across 295 models spanning three distinct tasks and three hardware platforms demonstrates a median prediction error of only 19.6%, substantially outperforming current state-of-the-art methods.

Abstract

The widespread adoption of Artificial Intelligence (AI) has led to increasing concerns about energy consumption, yet there is a lack of standardized methodologies to accurately estimate AI inference energy consumption, particularly across various tasks and architectures. In this study, we propose a task independent, layer-wise energy estimation model for AI architectures. Our model is evaluated on a large dataset of more than 100,000 layers for 295 neural network architectures across 3 widely-used tasks and 3 distinct hardware platforms. Our approach achieves a median error of 19.6%, outperforming state-of-the-art methods. We further show that layer-wise decomposition generalize to new tasks without complete retraining, by leveraging shared layers across architectures. It offer tools, insights and a precise methodology to empower stakeholders in designing energy-efficient AI systems.

Problem

• energy estimation
• neural networks
• inference energy
• AI energy consumption
• layer-wise decomposition

Innovation

• layer-wise energy estimation
• task-independent modeling
• neural network inference
• energy-efficient AI
• cross-architecture generalization