Semi-rPPG: Semi-Supervised Remote Physiological Measurement with Curriculum Pseudo-Labeling

2025-02-06

AI Summary

To address the scarcity of labeled videos and poor model generalization in remote photoplethysmography (rPPG)-based heart rate estimation, this paper proposes Semi-rPPG, a semi-supervised framework. Methodologically, it introduces two key innovations: (1) a novel curriculum-style pseudo-labeling strategy that adaptively selects high-quality unlabeled samples based on signal-to-noise ratio (SNR) estimation; and (2) a consistency regularization scheme tailored to quasi-periodic physiological signals, integrated with video temporal augmentations to enhance robustness in temporal modeling. Evaluated on four public datasets under both cross-domain and intra-dataset semi-supervised benchmarks, Semi-rPPG significantly outperforms three representative baseline methods. Ablation studies confirm the effectiveness of both the SNR-guided curriculum learning and the physiological-signal-aware consistency regularization. Overall, Semi-rPPG establishes a scalable and interpretable paradigm for low-resource rPPG modeling.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Computer Vision: Multi-modal Vision
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Remote Photoplethysmography (rPPG) is a promising technique to monitor physiological signals such as heart rate from facial videos. However, the labeled facial videos in this research are challenging to collect. Current rPPG research is mainly based on several small public datasets collected in simple environments, which limits the generalization and scale of the AI models. Semi-supervised methods that leverage a small amount of labeled data and abundant unlabeled data can fill this gap for rPPG learning. In this study, a novel semi-supervised learning method named Semi-rPPG that combines curriculum pseudo-labeling and consistency regularization is proposed to extract intrinsic physiological features from unlabelled data without impairing the model from noises. Specifically, a curriculum pseudo-labeling strategy with signal-to-noise ratio (SNR) criteria is proposed to annotate the unlabelled data while adaptively filtering out the low-quality unlabelled data. Besides, a novel consistency regularization term for quasi-periodic signals is proposed through weak and strong augmented clips. To benefit the research on semi-supervised rPPG measurement, we establish a novel semi-supervised benchmark for rPPG learning through intra-dataset and cross-dataset evaluation on four public datasets. The proposed Semi-rPPG method achieves the best results compared with three classical semi-supervised methods under different protocols. Ablation studies are conducted to prove the effectiveness of the proposed methods.

Problem

• Improves remote heart rate monitoring accuracy
• Addresses limited labeled facial video datasets
• Enhances semi-supervised learning for physiological signals

Innovation

• Semi-supervised learning for rPPG
• Curriculum pseudo-labeling strategy
• Consistency regularization term