THOR: A Generic Energy Estimation Approach for On-Device Training

2025-01-27

AI Summary

Energy estimation for on-device AI training on battery-powered mobile devices—such as smartphones, AR/VR headsets, and IoT endpoints—suffers from low accuracy and poor generalizability under heterogeneous hardware and complex DNN models. This work first uncovers the additive property of layer-wise energy consumption during DNN training. Leveraging this insight, we propose a universal, cross-model and cross-platform energy modeling framework based on fine-grained layer-level energy decomposition and Gaussian Process Regression (GP). The framework significantly improves estimation robustness, reducing the mean absolute percentage error (MAPE) by up to 30% across multiple DNN models and diverse hardware platforms. Furthermore, it enables energy-aware pruning, cutting training energy consumption by 50%. Our approach establishes a scalable foundation for energy modeling and optimization, advancing green on-device AI.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Natural Language Processing: Learning & Optimization for NLP
• Computer Vision: Learning & Optimization for CV

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Battery-powered mobile devices (e.g., smartphones, AR/VR glasses, and various IoT devices) are increasingly being used for AI training due to their growing computational power and easy access to valuable, diverse, and real-time data. On-device training is highly energy-intensive, making accurate energy consumption estimation crucial for effective job scheduling and sustainable AI. However, the heterogeneity of devices and the complexity of models challenge the accuracy and generalizability of existing estimation methods. This paper proposes THOR, a generic approach for energy consumption estimation in deep neural network (DNN) training. First, we examine the layer-wise energy additivity property of DNNs and strategically partition the entire model into layers for fine-grained energy consumption profiling. Then, we fit Gaussian Process (GP) models to learn from layer-wise energy consumption measurements and estimate a DNN's overall energy consumption based on its layer-wise energy additivity property. We conduct extensive experiments with various types of models across different real-world platforms. The results demonstrate that THOR has effectively reduced the Mean Absolute Percentage Error (MAPE) by up to 30%. Moreover, THOR is applied in guiding energy-aware pruning, successfully reducing energy consumption by 50%, thereby further demonstrating its generality and potential.

Problem

• Mobile Devices
• AI Model Training
• Power Consumption

Innovation

• THOR
• Energy Estimation
• Deep Learning Training