OnDA: On-device Channel Pruning for Efficient Personalized Keyword Spotting

2026-02-25

AI Summary

This work addresses the distribution shift in keyword spotting (KWS) tasks caused by variations across users and environments by proposing an on-device joint optimization framework that uniquely integrates weight adaptation with online structured channel pruning. This approach enables personalized model compression and efficient inference by combining data-agnostic and data-aware pruning criteria within a self-learning pipeline tailored to individual users. The system supports online training and inference directly on embedded GPUs. Evaluated on the HeySnips and HeySnapdragon datasets, the method achieves a model compression ratio of 9.63×. On the Jetson Orin Nano platform, it reduces training and inference latency by 1.52× and 1.64×, respectively, while decreasing energy consumption by 1.57× and 1.77×.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Distributed Search

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

Always-on keyword spotting (KWS) demands on-device adaptation to cope with user- and environment-specific distribution shifts under tight latency and energy budgets. This paper proposes, for the first time, coupling weight adaptation (i.e., on-device training) with architectural adaptation, in the form of online structured channel pruning, for personalized on-device KWS. Starting from a state-of-the-art self-learning personalized KWS pipeline, we compare data-agnostic and data-aware pruning criteria applied on in-field pseudo-labelled user data. On the HeySnips and HeySnapdragon datasets, we achieve up to 9.63x model-size compression with respect to unpruned baselines at iso-task performance, measured as the accuracy at 0.5 false alarms per hour. When deploying our adaptation pipeline on a Jetson Orin Nano embedded GPU, we achieve up to 1.52x/1.57x and 1.64x/1.77x latency and energy-consumption improvements during online training/inference compared to weights-only adaptation.

Problem

• keyword spotting
• on-device adaptation
• distribution shift
• latency constraint
• energy efficiency

Innovation

• on-device adaptation
• channel pruning
• personalized keyword spotting
• online structured pruning
• energy-efficient inference