DD-rPPGNet: De-interfering and Descriptive Feature Learning for Unsupervised rPPG Estimation

2024-07-31

arXiv.org

AI Summary

Existing unsupervised remote photoplethysmography (rPPG) methods suffer from limited heart rate estimation accuracy due to their failure to explicitly model inherent confounders in facial videos—such as motion, illumination variations, and sensor noise. To address this, we propose an unsupervised confounder disentanglement and suppression framework. First, we introduce a learnable 3D Descriptive Convolution (3DLDC) to precisely capture subtle spatiotemporal chrominance variations. Second, we design a two-stage confounder modeling mechanism that jointly leverages contrastive learning and local spatiotemporal similarity constraints to explicitly disentangle rPPG-relevant features from confounder-corrupted ones. Crucially, our method operates entirely without physiological ground-truth labels. Evaluated on five benchmark datasets, it consistently outperforms state-of-the-art unsupervised approaches, achieving a mean absolute error of only ±1.82 BPM in heart rate estimation—on par with leading supervised methods. This work establishes a new paradigm for label-free rPPG analysis.

Technology Category

• Machine Learning: Unsupervised & Self-Supervised Learning
• Computer Vision: Video Understanding & Activity Analysis
• Search and Optimization: Non-convex Optimization

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Remote Photoplethysmography (rPPG) aims to measure physiological signals and Heart Rate (HR) from facial videos. Recent unsupervised rPPG estimation methods have shown promising potential in estimating rPPG signals from facial regions without relying on ground truth rPPG signals. However, these methods seem oblivious to interference existing in rPPG signals and still result in unsatisfactory performance. In this paper, we propose a novel De-interfered and Descriptive rPPG Estimation Network (DD-rPPGNet) to eliminate the interference within rPPG features for learning genuine rPPG signals. First, we investigate the characteristics of local spatial-temporal similarities of interference and design a novel unsupervised model to estimate the interference. Next, we propose an unsupervised de-interfered method to learn genuine rPPG signals with two stages. In the first stage, we estimate the initial rPPG signals by contrastive learning from both the training data and their augmented counterparts. In the second stage, we use the estimated interference features to derive de-interfered rPPG features and encourage the rPPG signals to be distinct from the interference. In addition, we propose an effective descriptive rPPG feature learning by developing a strong 3D Learnable Descriptive Convolution (3DLDC) to capture the subtle chrominance changes for enhancing rPPG estimation. Extensive experiments conducted on five rPPG benchmark datasets demonstrate that the proposed DD-rPPGNet outperforms previous unsupervised rPPG estimation methods and achieves competitive performances with state-of-the-art supervised rPPG methods.

Problem

• Eliminates interference in rPPG signals
• Uses unsupervised learning for rPPG estimation
• Enhances rPPG feature learning with 3DLDC

Innovation

• Unsupervised de-interfered rPPG estimation
• Contrastive learning for initial signals
• 3D Learnable Descriptive Convolution