AvatarArtist: Open-Domain 4D Avatarization

2025-03-25

AI Summary

This work addresses the challenging cross-domain mapping problem of generating high-fidelity, temporally coherent 4D dynamic avatars from arbitrary-style portrait images—without requiring 3D annotations or video supervision. Methodologically, we introduce parametric tri-planes as a compact 4D intermediate representation and propose the first joint training paradigm that synergistically leverages GAN and 2D diffusion model priors. We construct a multi-domain image–tri-plane paired dataset and optimize via a combined adversarial-reconstruction loss. Our approach achieves fully unsupervised open-domain 4D avatar generation for the first time, demonstrating strong cross-domain robustness across heterogeneous portrait sources and delivering high geometric and appearance fidelity. To foster reproducibility and further research, we release our code, dataset, and pre-trained models publicly.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

This work focuses on open-domain 4D avatarization, with the purpose of creating a 4D avatar from a portrait image in an arbitrary style. We select parametric triplanes as the intermediate 4D representation and propose a practical training paradigm that takes advantage of both generative adversarial networks (GANs) and diffusion models. Our design stems from the observation that 4D GANs excel at bridging images and triplanes without supervision yet usually face challenges in handling diverse data distributions. A robust 2D diffusion prior emerges as the solution, assisting the GAN in transferring its expertise across various domains. The synergy between these experts permits the construction of a multi-domain image-triplane dataset, which drives the development of a general 4D avatar creator. Extensive experiments suggest that our model, AvatarArtist, is capable of producing high-quality 4D avatars with strong robustness to various source image domains. The code, the data, and the models will be made publicly available to facilitate future studies..

Problem

• Creating 4D avatars from arbitrary style portrait images
• Bridging images and triplanes using GANs and diffusion models
• Developing a general 4D avatar creator with multi-domain robustness

Innovation

• Parametric triplanes as 4D representation
• GANs and diffusion models combined
• Multi-domain image-triplane dataset creation