Fine-Grained 3D Facial Reconstruction for Micro-Expressions

2026-03-07

AI Summary

Existing 3D facial reconstruction methods struggle to accurately capture subtle, brief, and low-intensity micro-expressions, leading to significant challenges in feature extraction. To address this limitation, this work proposes a fine-grained micro-expression reconstruction approach that integrates a plug-and-play dynamic encoding module with a dynamics-guided local mesh deformation mechanism. The method effectively fuses multi-source cues—including optical flow, facial landmarks, and 3D geometry—and leverages transfer learning from macro-expression data to mitigate the scarcity of micro-expression training samples. This approach achieves the first high-fidelity 3D micro-expression reconstruction, significantly outperforming existing methods across multiple datasets and setting new state-of-the-art results in both geometric accuracy and perceptual detail.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Learning with Manifolds
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

Recent advances in 3D facial expression reconstruction have demonstrated remarkable performance in capturing macro-expressions, yet the reconstruction of micro-expressions remains unexplored. This novel task is particularly challenging due to the subtle, transient, and low-intensity nature of micro-expressions, which complicate the extraction of stable and discriminative features essential for accurate reconstruction. In this paper, we propose a fine-grained micro-expression reconstruction method that integrates a global dynamic feature capturing stable facial motion patterns with a locally-enriched feature incorporating multiple informative cues from 2D motions, facial priors and 3D facial geometry. Specifically, we devise a plug-and-play dynamic-encoded module to extract micro-expression feature for global facial action, allowing it to leverage prior knowledge from abundant macro-expression data to mitigate the scarcity of micro-expression data. Subsequently, a dynamic-guided mesh deformation module is designed for extracting aggregated local features from dense optical flow, sparse landmark cues and facial mesh geometry, which adaptively refines fine-grained facial micro-expression without compromising global 3D geometry. Extensive experiments on micro-expression datasets demonstrate that our method consistently outperforms state-of-the-art methods in both geometric accuracy and perceptual detail.

Problem

• micro-expressions
• 3D facial reconstruction
• fine-grained reconstruction
• facial motion
• feature extraction

Innovation

• micro-expression reconstruction
• fine-grained 3D facial modeling
• dynamic-encoded module
• mesh deformation
• multi-cue fusion