🤖 AI Summary
This work addresses the challenge of setting the gradient estimation sample size \( q \) in zeroth-order optimization on neural processing unit (NPU)-based microcontrollers, where small \( q \) introduces high noise and premature convergence, while large \( q \) incurs excessive computational overhead—both traditionally requiring costly hyperparameter tuning. To overcome this, the paper proposes QScheduler, an adaptive algorithm that dynamically adjusts \( q \) during training without any pre-specified or tuned values. QScheduler enables, for the first time, INT8-quantized zeroth-order on-device training on the STM32N6 Neural-ART NPU. Experiments on EuroSAT and STL-10 demonstrate that QScheduler achieves accuracy comparable to meticulously tuned fixed-\( q \) baselines while substantially reducing hyperparameter tuning costs, facilitating efficient deployment of models such as ResNet18 and MobileNetV2.
📝 Abstract
Zeroth-Order (ZO) optimization enables On-Device Learning (ODL) on NPU-equipped microcontrollers by estimating gradients through forward passes alone, bypassing the need for backpropagation primitives and reducing memory requirements. The number of gradient samples q critically affects training: insufficient samples produce noisy gradients that plateau early, while excessive samples consume more computational resources. However, finding an optimal q typically requires costly hyperparameter searches. This work introduces QScheduler, an adaptive algorithm that adjusts q based on training progress, and provides the first proof-of-concept of INT8 quantized on-device training on the STM32N6's Neural-ART NPU. Experiments on EuroSAT and STL-10 show that QScheduler matches well-tuned fixed-q configurations for both ResNet18 and MobileNetV2, without requiring prior q hyperparameter optimization.