Few-Shot Identity Adaptation for 3D Talking Heads via Global Gaussian Field

2025-06-27

AI Summary

Existing talking-head synthesis methods rely on identity-specific models, requiring full retraining for each new subject—entailing high computational cost and poor scalability. To address this, we propose a 3D talking-head synthesis framework built upon a Global Gaussian Field (GGF) and a Universal Motion Field (UMF). The GGF encodes identity-agnostic, shared static 3D structure, while the UMF captures cross-identity motion priors as a universal dynamic representation. Jointly, they enable rapid adaptation to unseen identities from only a few seconds of reference video. Our framework significantly improves few-shot identity generalization and fine-tuning efficiency, achieving state-of-the-art performance in visual fidelity, identity preservation, and adaptation speed.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: 3D Computer Vision

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Reconstruction and rendering-based talking head synthesis methods achieve high-quality results with strong identity preservation but are limited by their dependence on identity-specific models. Each new identity requires training from scratch, incurring high computational costs and reduced scalability compared to generative model-based approaches. To overcome this limitation, we propose FIAG, a novel 3D speaking head synthesis framework that enables efficient identity-specific adaptation using only a few training footage. FIAG incorporates Global Gaussian Field, which supports the representation of multiple identities within a shared field, and Universal Motion Field, which captures the common motion dynamics across diverse identities. Benefiting from the shared facial structure information encoded in the Global Gaussian Field and the general motion priors learned in the motion field, our framework enables rapid adaptation from canonical identity representations to specific ones with minimal data. Extensive comparative and ablation experiments demonstrate that our method outperforms existing state-of-the-art approaches, validating both the effectiveness and generalizability of the proposed framework. Code is available at: extit{https://github.com/gme-hong/FIAG}.

Problem

• Limited scalability of identity-specific 3D talking head models
• High computational cost for training new identities from scratch
• Need for efficient few-shot adaptation in talking head synthesis

Innovation

• Global Gaussian Field for multi-identity representation
• Universal Motion Field for common dynamics
• Few-shot adaptation from canonical identity