FIELDS: Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision

2025-11-26

AI Summary

Existing 3D face reconstruction methods rely heavily on 2D supervision and lack authentic 3D expression annotations, limiting their ability to model subtle affective details. To address this, we propose a dual-supervised high-fidelity reconstruction framework: (1) leveraging real 4D scan-driven 3D expression parameter supervision to guide geometric fidelity, and (2) introducing an intensity-aware auxiliary emotion recognition branch to jointly optimize geometric and semantic consistency of expressions. A self-supervised 2D image reconstruction loss further enhances generalization, effectively bridging the 2D–3D domain gap and mitigating expression intensity bias. Evaluated on in-the-wild single-image inputs, our method significantly improves emotion recognition accuracy while generating 3D face models with both high geometric precision and perceptually authentic expressions. To the best of our knowledge, this is the first end-to-end, interpretable emotion-driven 3D face reconstruction approach.

Technology Category

• Computer Vision: 3D Computer Vision
• Cognitive Modeling & Cognitive Systems: Affective Computing
• Machine Learning: Multimodal Learning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Facial expressions convey the bulk of emotional information in human communication, yet existing 3D face reconstruction methods often miss subtle affective details due to reliance on 2D supervision and lack of 3D ground truth. We propose FIELDS (Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision) to address these limitations by extending self-supervised 2D image consistency cues with direct 3D expression parameter supervision and an auxiliary emotion recognition branch. Our encoder is guided by authentic expression parameters from spontaneous 4D facial scans, while an intensity-aware emotion loss encourages the 3D expression parameters to capture genuine emotion content without exaggeration. This dual-supervision strategy bridges the 2D/3D domain gap and mitigates expression-intensity bias, yielding high-fidelity 3D reconstructions that preserve subtle emotional cues. From a single image, FIELDS produces emotion-rich face models with highly realistic expressions, significantly improving in-the-wild facial expression recognition performance without sacrificing naturalness.

Problem

• Reconstructing 3D faces with subtle emotional details
• Bridging 2D/3D domain gap in facial expression modeling
• Enhancing emotion recognition while preserving natural facial expressions

Innovation

• Direct 3D expression parameter supervision from 4D scans
• Auxiliary emotion recognition branch with intensity-aware loss
• Dual-supervision strategy bridging 2D/3D domain gap