Skullptor: High Fidelity 3D Head Reconstruction in Seconds with Multi-View Normal Prediction

2026-02-24

AI Summary

Existing high-fidelity 3D face reconstruction methods often rely on either a large number of camera views or substantial computational resources, making it challenging to balance efficiency with geometric detail. This work proposes a hybrid reconstruction framework that, for the first time, integrates cross-view attention into multi-view surface normal prediction to produce consistent and fine-grained geometric priors. These priors are then incorporated into an inverse rendering optimization pipeline, effectively combining feedforward inference with iterative refinement. The proposed approach achieves reconstruction quality comparable to dense photogrammetry while significantly reducing both the required number of input views and computational overhead, outperforming current state-of-the-art methods in both single-image and multi-view settings.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Reconstructing high-fidelity 3D head geometry from images is critical for a wide range of applications, yet existing methods face fundamental limitations. Traditional photogrammetry achieves exceptional detail but requires extensive camera arrays (25-200+ views), substantial computation, and manual cleanup in challenging areas like facial hair. Recent alternatives present a fundamental trade-off: foundation models enable efficient single-image reconstruction but lack fine geometric detail, while optimization-based methods achieve higher fidelity but require dense views and expensive computation. We bridge this gap with a hybrid approach that combines the strengths of both paradigms. Our method introduces a multi-view surface normal prediction model that extends monocular foundation models with cross-view attention to produce geometrically consistent normals in a feed-forward pass. We then leverage these predictions as strong geometric priors within an inverse rendering optimization framework to recover high-frequency surface details. Our approach outperforms state-of-the-art single-image and multi-view methods, achieving high-fidelity reconstruction on par with dense-view photogrammetry while reducing camera requirements and computational cost. The code and model will be released.

Problem

• 3D head reconstruction
• high fidelity
• multi-view
• surface normal prediction
• geometric detail

Innovation

• multi-view normal prediction
• hybrid 3D reconstruction
• cross-view attention
• inverse rendering optimization
• high-fidelity head modeling