Not Only Consistency: Enhance Test-Time Adaptation with Spatio-temporal Inconsistency for Remote Physiological Measurement

2025-07-10

AI Summary

Remote photoplethysmography (rPPG) faces challenges in real-world deployment due to privacy constraints and strict latency requirements, rendering conventional domain adaptation methods inapplicable under unknown target environments. Method: This paper proposes CiCi, a novel test-time adaptation (TTA) framework grounded in self-supervision. CiCi introduces, for the first time, a dual prior—“frequency-domain consistency” and “time-domain inconsistency”—to encode expert physiological knowledge into a label-free, source-data-free self-supervised mechanism. It further incorporates a gradient dynamic regulation strategy to resolve conflicts among multiple priors. Contribution/Results: CiCi requires no source-domain data, no ground-truth labels, and preserves data privacy while enabling lightweight online updates. Evaluated across five heterogeneous datasets, it significantly outperforms existing TTA and domain generalization methods, achieving state-of-the-art real-time adaptive performance.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Security and Privacy: Privacy-enhancing technologies
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Remote photoplethysmography (rPPG) has emerged as a promising non-invasive method for monitoring physiological signals using the camera. Although various domain adaptation and generalization methods were proposed to promote the adaptability of deep-based rPPG models in unseen deployment environments, considerations in aspects like privacy concerns and real-time adaptation restrict their application in real-world deployment. Thus, we aim to propose a novel fully Test-Time Adaptation (TTA) strategy tailored for rPPG tasks in this work. Specifically, based on prior knowledge in physiology and our observations, we noticed not only there is spatio-temporal consistency in the frequency domain of rPPG signals, but also that inconsistency in the time domain was significant. Given this, by leveraging both consistency and inconsistency priors, we introduce an innovative expert knowledge-based self-supervised extbf{C}onsistency- extbf{i}n extbf{C}onsistency- extbf{i}ntegration ( extbf{CiCi}) framework to enhances model adaptation during inference. Besides, our approach further incorporates a gradient dynamic control mechanism to mitigate potential conflicts between priors, ensuring stable adaptation across instances. Through extensive experiments on five diverse datasets under the TTA protocol, our method consistently outperforms existing techniques, presenting state-of-the-art performance in real-time self-supervised adaptation without accessing source data. The code will be released later.

Problem

• Enhancing rPPG model adaptability in unseen environments
• Leveraging spatio-temporal consistency and inconsistency for TTA
• Mitigating conflicts between priors for stable adaptation

Innovation

• Leverages spatio-temporal inconsistency for rPPG
• Introduces CiCi framework for self-supervised adaptation
• Uses gradient control to stabilize adaptation