Bringing Your Portrait to 3D Presence

2025-11-27

AI Summary

To address three key bottlenecks in animatable 3D human avatar reconstruction from a single portrait image—pose and composition sensitivity, scarcity of training data, and instability in proxy mesh estimation—this paper proposes the first unified framework. Methodologically, it introduces: (1) a Dual-UV representation that decouples appearance and geometry via Core-UV and Shell-UV branches, eliminating pose- and composition-induced feature shifts; (2) a generative synthetic data manifold integrating GAN-driven diverse 2D renderings with geometrically consistent 3D synthesis; and (3) a visibility-aware robust proxy mesh tracking mechanism. Trained solely on synthetic upper-body data, our method achieves state-of-the-art performance on head and upper-body reconstruction, while delivering highly competitive results for full-body reconstruction. Crucially, it significantly improves generalization to in-the-wild scenarios, demonstrating strong robustness under unconstrained conditions.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

We present a unified framework for reconstructing animatable 3D human avatars from a single portrait across head, half-body, and full-body inputs. Our method tackles three bottlenecks: pose- and framing-sensitive feature representations, limited scalable data, and unreliable proxy-mesh estimation. We introduce a Dual-UV representation that maps image features to a canonical UV space via Core-UV and Shell-UV branches, eliminating pose- and framing-induced token shifts. We also build a factorized synthetic data manifold combining 2D generative diversity with geometry-consistent 3D renderings, supported by a training scheme that improves realism and identity consistency. A robust proxy-mesh tracker maintains stability under partial visibility. Together, these components enable strong in-the-wild generalization. Trained only on half-body synthetic data, our model achieves state-of-the-art head and upper-body reconstruction and competitive full-body results. Extensive experiments and analyses further validate the effectiveness of our approach.

Problem

• Reconstructs animatable 3D avatars from single portraits
• Addresses pose-sensitive features and limited scalable data
• Improves proxy-mesh reliability for in-the-wild generalization

Innovation

• Dual-UV representation maps features to canonical UV space
• Factorized synthetic data combines 2D diversity with 3D consistency
• Robust proxy-mesh tracker maintains stability under partial visibility