Align the GAP: Prior-based Unified Multi-Task Remote Physiological Measurement Framework For Domain Generalization and Personalization

2025-06-19

AI Summary

This paper addresses the long-standing challenge of jointly optimizing multi-source same-semantics domain generalization (MSSDG) and test-time personalized adaptation (TTPA) in multi-task remote photoplethysmography (rPPG). We propose the first unified framework that (1) disentangles facial video representations into invariant physiological semantics, subject-specific biases, and environmental noise; (2) integrates physiological priors with rPPG feature constraints to enable robust cross-domain estimation; and (3) introduces a lightweight meta-adaptation mechanism for real-time TTPA. Our approach bridges the conceptual and methodological gap between MSSDG and TTPA for the first time. Extensive experiments on six public benchmarks and a newly collected real-world driving dataset demonstrate significant improvements in both generalization performance (e.g., heart rate, respiration rate estimation) across domains and personalized accuracy under unseen conditions. The code and the new driving dataset will be publicly released.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Multi-modal Vision
• Application Domains: Mobility, Driving & Flight

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Large-scale security measurements

Abstract

Multi-source synsemantic domain generalization (MSSDG) for multi-task remote physiological measurement seeks to enhance the generalizability of these metrics and attracts increasing attention. However, challenges like partial labeling and environmental noise may disrupt task-specific accuracy. Meanwhile, given that real-time adaptation is necessary for personalized products, the test-time personalized adaptation (TTPA) after MSSDG is also worth exploring, while the gap between previous generalization and personalization methods is significant and hard to fuse. Thus, we proposed a unified framework for MSSD extbf{G} and TTP extbf{A} employing extbf{P}riors ( extbf{GAP}) in biometrics and remote photoplethysmography (rPPG). We first disentangled information from face videos into invariant semantics, individual bias, and noise. Then, multiple modules incorporating priors and our observations were applied in different stages and for different facial information. Then, based on the different principles of achieving generalization and personalization, our framework could simultaneously address MSSDG and TTPA under multi-task remote physiological estimation with minimal adjustments. We expanded the MSSDG benchmark to the TTPA protocol on six publicly available datasets and introduced a new real-world driving dataset with complete labeling. Extensive experiments that validated our approach, and the codes along with the new dataset will be released.

Problem

• Enhance multi-task remote physiological measurement generalizability across domains
• Address partial labeling and noise challenges in task-specific accuracy
• Bridge gap between domain generalization and test-time personalization adaptation

Innovation

• Disentangled facial video information into three components
• Employed priors for multi-task generalization and personalization
• Expanded benchmark with new real-world driving dataset