ToTMNet: FFT-Accelerated Toeplitz Temporal Mixing Network for Lightweight Remote Photoplethysmography

2026-01-07

arXiv.org

AI Summary

This work proposes ToTMNet, a lightweight neural network addressing the high computational cost, large parameter count, and complex temporal modeling of existing deep learning approaches in remote photoplethysmography (rPPG). ToTMNet uniquely integrates Toeplitz-structured matrices with a temporal mixing mechanism, leveraging recurrent embeddings and FFT acceleration to achieve near-linear complexity for global temporal modeling. A gating mechanism is further introduced to enhance cross-domain robustness. With only 63k parameters, the model achieves a mean absolute error (MAE) of 1.055 bpm and a Pearson correlation coefficient of 0.996 on UBFC-rPPG. Under the challenging SCAMPS→UBFC cross-domain setting, it attains an MAE of 1.582 bpm and a correlation of 0.994, demonstrating an exceptional balance between efficiency and performance.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Temporal Planning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Remote photoplethysmography (rPPG) estimates a blood volume pulse (BVP) waveform from facial videos captured by commodity cameras. Although recent deep models improve robustness compared to classical signal-processing approaches, many methods increase computational cost and parameter count, and attention-based temporal modeling introduces quadratic scaling with respect to the temporal length. This paper proposes ToTMNet, a lightweight rPPG architecture that replaces temporal attention with an FFT-accelerated Toeplitz temporal mixing layer. The Toeplitz operator provides full-sequence temporal receptive field using a linear number of parameters in the clip length and can be applied in near-linear time using circulant embedding and FFT-based convolution. ToTMNet integrates the global Toeplitz temporal operator into a compact gated temporal mixer that combines a local depthwise temporal convolution branch with gated global Toeplitz mixing, enabling efficient long-range temporal filtering while only having 63k parameters. Experiments on two datasets, UBFC-rPPG (real videos) and SCAMPS (synthetic videos), show that ToTMNet achieves strong heart-rate estimation accuracy with a compact design. On UBFC-rPPG intra-dataset evaluation, ToTMNet reaches 1.055 bpm MAE with Pearson correlation 0.996. In a synthetic-to-real setting (SCAMPS to UBFC-rPPG), ToTMNet reaches 1.582 bpm MAE with Pearson correlation 0.994. Ablation results confirm that the gating mechanism is important for effectively using global Toeplitz mixing, especially under domain shift. The main limitation of this preprint study is the use of only two datasets; nevertheless, the results indicate that Toeplitz-structured temporal mixing is a practical and efficient alternative to attention for rPPG.

Problem

• remote photoplethysmography
• lightweight
• temporal modeling
• computational efficiency
• attention mechanism

Innovation

• Toeplitz temporal mixing
• FFT acceleration
• lightweight rPPG
• gated temporal mixer
• circulant embedding