Accelerated Training on Low-Power Edge Devices

2025-02-25

AI Summary

To address slow training on power-constrained low-power edge devices, this paper proposes a cross-layer joint optimization framework that co-optimizes GPU frequency and batch size. The method integrates a batch-size efficiency prediction model with dynamic, real-time device power measurement modeling, enabling simultaneous minimization of both training time and energy consumption under strict power budgets—a first in the literature. Through system-level parameter coordination and hardware-in-the-loop runtime optimization, it achieves a 2.4× speedup in training latency and substantial energy reduction on real edge platforms, without compromising model accuracy. The core innovation lies in deeply embedding hardware power characteristics into the training configuration decision loop, thereby transcending conventional static hyperparameter tuning paradigms.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Training on edge devices poses several challenges as these devices are generally resource-constrained, especially in terms of power. State-of-the-art techniques at the device level reduce the GPU frequency to enforce power constraints, leading to a significant increase in training time. To accelerate training, we propose to jointly adjust the system and application parameters (in our case, the GPU frequency and the batch size of the training task) while adhering to the power constraints on devices. We introduce a novel cross-layer methodology that combines predictions of batch size efficiency and device profiling to achieve the desired optimization. Our evaluation on real hardware shows that our method outperforms the current baselines that depend on state of the art techniques, reducing the training time by $2.4 imes$ with results very close to optimal. Our measurements also indicate a substantial reduction in the overall energy used for the training process. These gains are achieved without reduction in the performance of the trained model.

Problem

• Accelerate training on edge devices
• Optimize GPU frequency and batch size
• Reduce training time and energy usage

Innovation

• Jointly adjust system and application parameters
• Introduce novel cross-layer methodology
• Reduce training time and energy usage