3D Face Reconstruction with Geometry Details from a Single Color Image Under Occluded Scenes

2024-12-25

International Conference on Artificial Neural Networks

AI Summary

To address the degradation in 3D face reconstruction quality under severe occlusions (e.g., glasses, masks, or hands) in single-color images, this paper proposes an occlusion-robust, high-fidelity facial geometry reconstruction method. Our approach introduces two key innovations: (1) the first occlusion-aware geometric prior, integrated with a local-global decoupled reconstruction framework to achieve semantically consistent geometric completion of occluded regions; and (2) an implicit neural representation (INR)-based differentiable rendering pipeline, enhanced by multi-scale feature distillation and an occlusion-robust loss function. Evaluated on NoW, MICC, and our newly introduced OccludedFace dataset, the method achieves state-of-the-art performance: it reduces average geometric error by 32% and improves landmark reconstruction accuracy by 27% compared to prior work.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Problem

• Partially Occluded Faces
• Color Photo Utilization
• High-quality 3D Reconstruction

Innovation

• 3D face reconstruction
• occlusion handling
• bump mapping technique