CardiacMamba: Fair and Robust RGB-RF Fusion for Remote Heart Rate Estimation via State Space Modeling

2026-08-16

AI Summary

This study addresses the fairness limitations of RGB-based remote photoplethysmography caused by environmental interference. We propose an RGB-RF fusion framework leveraging state-space models to integrate optical and radio-frequency signals. Specifically, temporal difference Mamba enhances RF micro-motion features, while a bidirectional SSM aligns heterogeneous modal dynamics, complemented by channel-wise FFT for spectral refinement. This architecture enables precise fusion of photoplethysmographic signals with cardiac motion cues. Evaluated on the EquiPleth dataset, the method achieves state-of-the-art performance with a mean absolute error (MAE) of 0.96 bpm and reduces cross-skin-tone MAE disparity to 0.26 bpm. Furthermore, it demonstrates significant robustness under degraded conditions such as modality missingness. These results effectively enhance both the fairness and reliability of non-contact heart rate estimation.

Abstract

Remote photoplethysmography (rPPG) enables non-contact heart rate (HR) monitoring from facial videos, but RGB-only methods are vulnerable to illumination changes, motion artifacts, and skin-tone-dependent optical reflectance. We propose CardiacMamba, a fair and robust RGB-RF fusion framework that integrates optical facial cues and radio-frequency cardiac motion cues through state space modeling. CardiacMamba introduces a Temporal Difference Mamba Module (TDMM) to enhance subtle RF temporal variations, a bidirectional SSM-based interaction mechanism to align heterogeneous RGB-RF dynamics, and a Channel-wise Fast Fourier Transform (CFFT) module for channel-domain spectral refinement. On the EquiPleth dataset, CardiacMamba achieves state-of-the-art performance with 0.96 bpm MAE, 3.06 bpm RMSE, and 0.97 Pearson correlation, while reducing the observed light-dark skin-tone MAE gap to 0.26 bpm and maintaining robustness under RGB degradation and RF-missing conditions

Problem

• Remote Heart Rate Estimation
• Fairness
• Robustness
• RGB-RF Fusion

Innovation

• State Space Modeling
• RGB-RF Fusion
• Temporal Difference Mamba Module
• Bidirectional SSM Interaction
• Channel-wise FFT