Towards Characterizing Knowledge Distillation of PPG Heart Rate Estimation Models

2025-11-24

AI Summary

This work addresses the challenge of deploying large pre-trained photoplethysmography (PPG)-based heart rate estimation models on memory- and latency-constrained wearable edge devices. We propose a systematic knowledge distillation framework, comparatively evaluating hard distillation, soft distillation, decoupled knowledge distillation (DKD), and feature distillation across diverse teacher–student capacity configurations. For the first time, we quantitatively characterize the scaling relationship between model size and heart rate estimation accuracy, establishing a predictive lightweight design paradigm. Experimental results demonstrate that the optimal distillation strategy reduces student model parameters by over 80%, achieves inference latency under 15 ms, and maintains a mean absolute error of less than 2.5 BPM. This study provides both theoretical foundations and practical guidelines for efficient edge deployment of physiological signal perception models.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Learning & Optimization for CV
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Heart rate estimation from photoplethysmography (PPG) signals generated by wearable devices such as smartwatches and fitness trackers has significant implications for the health and well-being of individuals. Although prior work has demonstrated deep learning models with strong performance in the heart rate estimation task, in order to deploy these models on wearable devices, these models must also adhere to strict memory and latency constraints. In this work, we explore and characterize how large pre-trained PPG models may be distilled to smaller models appropriate for real-time inference on the edge. We evaluate four distillation strategies through comprehensive sweeps of teacher and student model capacities: (1) hard distillation, (2) soft distillation, (3) decoupled knowledge distillation (DKD), and (4) feature distillation. We present a characterization of the resulting scaling laws describing the relationship between model size and performance. This early investigation lays the groundwork for practical and predictable methods for building edge-deployable models for physiological sensing.

Problem

• Compressing large PPG heart rate models for wearable device deployment
• Evaluating distillation strategies to meet memory and latency constraints
• Characterizing performance trade-offs between model size and accuracy

Innovation

• Knowledge distillation for PPG heart rate estimation
• Four distillation strategies compared for edge deployment
• Characterizing scaling laws between model size and performance