Facial Spatiotemporal Graphs: Leveraging the 3D Facial Surface for Remote Physiological Measurement

2026-01-20

AI Summary

Existing remote photoplethysmography (rPPG) methods fail to explicitly align their receptive fields with the 3D facial geometry, leading to inaccurate physiological signal modeling. This work addresses this limitation by introducing the 3D facial surface as a strong prior for rPPG estimation for the first time. The authors propose a spatiotemporal graph (STGraph) representation that jointly encodes color and structural information through a sequence of 3D meshes, and design MeshPhys—a lightweight spatiotemporal graph convolutional network—that performs spatiotemporal modeling on surface-aligned nodes. The approach significantly enhances model interpretability, robustness, and generalization, achieving state-of-the-art or competitive performance across four benchmark datasets. Ablation studies further validate that surface constraints and 3D-aware features are crucial for robust physiological signal estimation.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Facial remote photoplethysmography (rPPG) methods estimate physiological signals by modeling subtle color changes on the 3D facial surface over time. However, existing methods fail to explicitly align their receptive fields with the 3D facial surface-the spatial support of the rPPG signal. To address this, we propose the Facial Spatiotemporal Graph (STGraph), a novel representation that encodes facial color and structure using 3D facial mesh sequences-enabling surface-aligned spatiotemporal processing. We introduce MeshPhys, a lightweight spatiotemporal graph convolutional network that operates on the STGraph to estimate physiological signals. Across four benchmark datasets, MeshPhys achieves state-of-the-art or competitive performance in both intra- and cross-dataset settings. Ablation studies show that constraining the model's receptive field to the facial surface acts as a strong structural prior, and that surface-aligned, 3D-aware node features are critical for robustly encoding facial surface color. Together, the STGraph and MeshPhys constitute a novel, principled modeling paradigm for facial rPPG, enabling robust, interpretable, and generalizable estimation. Code is available at https://samcantrill.github.io/facial-stgraph-rppg/ .

Problem

• remote photoplethysmography
• 3D facial surface
• spatiotemporal modeling
• physiological signal estimation
• facial rPPG

Innovation

• Spatiotemporal Graph
• 3D Facial Mesh
• Remote Photoplethysmography
• Surface-Aligned Receptive Field
• Graph Convolutional Network