Revisiting DNN Training for Intermittently Powered Energy Harvesting Micro Computers

2024-08-25

arXiv.org

AI Summary

Deep neural network (DNN) training on energy-harvesting microcomputers is severely hindered by intermittent power supply, leading to inefficiency and instability. Method: This paper proposes an energy-aware dynamic training framework. It introduces the first device-energy co-modeling adaptive dropout mechanism, enabling proactive adaptation to power interruptions. Additionally, it jointly designs intermittency-aware quantization and lightweight training scheduling to co-optimize energy efficiency and model accuracy. Contribution/Results: It is the first work to explicitly model energy state and embed it into the DNN training process—breaking the conventional assumption of continuous power. Evaluated under realistic energy harvesting traces, the framework achieves 6–22% higher inference accuracy than state-of-the-art methods, with less than 5% increase in computational overhead. This significantly enhances the practicality and robustness of ultra-low-power edge AI systems.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The deployment of Deep Neural Networks in energy-constrained environments, such as Energy Harvesting Wireless Sensor Networks, presents unique challenges, primarily due to the intermittent nature of power availability. To address these challenges, this study introduces and evaluates a novel training methodology tailored for DNNs operating within such contexts. In particular, we propose a dynamic dropout technique that adapts to both the architecture of the device and the variability in energy availability inherent in energy harvesting scenarios. Our proposed approach leverages a device model that incorporates specific parameters of the network architecture and the energy harvesting profile to optimize dropout rates dynamically during the training phase. By modulating the network's training process based on predicted energy availability, our method not only conserves energy but also ensures sustained learning and inference capabilities under power constraints. Our preliminary results demonstrate that this strategy provides 6 to 22 percent accuracy improvements compared to the state of the art with less than 5 percent additional compute. This paper details the development of the device model, describes the integration of energy profiles with intermittency aware dropout and quantization algorithms, and presents a comprehensive evaluation of the proposed approach using real-world energy harvesting data.

Problem

• Energy-efficient Learning
• Wireless Sensor Networks
• Intermittent Power

Innovation

• Energy-efficient Dropout
• Dynamic Adjustment
• Harvesting-aware Training