ELITE: Efficient Gaussian Head Avatar from a Monocular Video via Learned Initialization and TEst-time Generative Adaptation

2026-01-15

AI Summary

Monocular video-based avatar modeling remains challenging due to the lack of geometric cues, and existing approaches often suffer from limitations in generalization, efficiency, or identity fidelity. This work proposes ELITE, a novel system that uniquely integrates 3D data priors with 2D generative priors. It employs a feedforward Mesh2Gaussian model to rapidly initialize a 3D Gaussian avatar and performs generative test-time adaptation by jointly leveraging real and synthetic images. Furthermore, ELITE introduces a rendering-guided single-step diffusion enhancer, circumventing the hallucinations and inefficiencies associated with multi-step denoising. The method achieves significantly improved visual quality under complex expressions, operates 60× faster than purely 2D generative approaches, and demonstrates strong in-the-wild generalization while preserving high identity fidelity.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We introduce ELITE, an Efficient Gaussian head avatar synthesis from a monocular video via Learned Initialization and TEst-time generative adaptation. Prior works rely either on a 3D data prior or a 2D generative prior to compensate for missing visual cues in monocular videos. However, 3D data prior methods often struggle to generalize in-the-wild, while 2D generative prior methods are computationally heavy and prone to identity hallucination. We identify a complementary synergy between these two priors and design an efficient system that achieves high-fidelity animatable avatar synthesis with strong in-the-wild generalization. Specifically, we introduce a feed-forward Mesh2Gaussian Prior Model (MGPM) that enables fast initialization of a Gaussian avatar. To further bridge the domain gap at test time, we design a test-time generative adaptation stage, leveraging both real and synthetic images as supervision. Unlike previous full diffusion denoising strategies that are slow and hallucination-prone, we propose a rendering-guided single-step diffusion enhancer that restores missing visual details, grounded on Gaussian avatar renderings. Our experiments demonstrate that ELITE produces visually superior avatars to prior works, even for challenging expressions, while achieving 60x faster synthesis than the 2D generative prior method.

Problem

• Gaussian avatar
• monocular video
• in-the-wild generalization
• identity hallucination
• efficient synthesis

Innovation

• Gaussian avatar
• learned initialization
• test-time adaptation
• single-step diffusion
• monocular video