GaussianHead: High-fidelity Head Avatars with Learnable Gaussian Derivation

2023-12-04

AI Summary

This paper addresses key challenges in high-fidelity 3D head modeling and animation—namely, difficulty in modeling dynamic geometry, complex texture representation, and low rendering efficiency. We propose an anisotropic 3D Gaussian-based avatar representation. Our core contributions are: (1) a learnable Gaussian derivation mechanism that generates multiple “clone” Gaussians via spatial transformation to compactly encode fine-grained head structures; (2) a Gaussian inheritance derivation strategy to accelerate training convergence; and (3) joint optimization of geometry and appearance through integration of a motion deformation field, multi-resolution triplane texture representation, and learnable spatial transformations. Extensive experiments demonstrate state-of-the-art performance across reconstruction accuracy, cross-identity reenactment, and novel-view synthesis—achieving significantly enhanced realism and controllability while maintaining real-time rendering efficiency.

Abstract

Constructing vivid 3D head avatars for given subjects and realizing a series of animations on them is valuable yet challenging. This paper presents GaussianHead, which models the actional human head with anisotropic 3D Gaussians. In our framework, a motion deformation field and multi-resolution tri-plane are constructed respectively to deal with the head's dynamic geometry and complex texture. Notably, we impose an exclusive derivation scheme on each Gaussian, which generates its multiple doppelgangers through a set of learnable parameters for position transformation. With this design, we can compactly and accurately encode the appearance information of Gaussians, even those fitting the head's particular components with sophisticated structures. In addition, an inherited derivation strategy for newly added Gaussians is adopted to facilitate training acceleration. Extensive experiments show that our method can produce high-fidelity renderings, outperforming state-of-the-art approaches in reconstruction, cross-identity reenactment, and novel view synthesis tasks. Our code is available at: https://github.com/chiehwangs/gaussian-head.

Problem

• Constructing vivid 3D head avatars for animation
• Modeling dynamic geometry and complex texture
• Achieving high-fidelity rendering and reconstruction

Innovation

• Models head with anisotropic 3D Gaussians
• Uses motion deformation and tri-plane
• Learnable parameters for position transformation