TeGA: Texture Space Gaussian Avatars for High-Resolution Dynamic Head Modeling

2025-05-08

AI Summary

Existing animatable 3D head models suffer from motion estimation errors and memory constraints, resulting in insufficient Gaussian primitives, loss of fine geometric details, and degraded 4K rendering quality. To address these limitations, we propose a high-fidelity dynamic head modeling framework featuring the first UVD tangent-space embedded deformable Gaussian representation. Our method introduces a UVD deformation field to enable dense Gaussian placement and precise modeling of high-frequency dynamics—including micro-expressions and skin wrinkles. Given multi-view video input, we jointly optimize Gaussian attributes and the UVD deformation field under a 3D Morphable Model (3DMM)-guided mesh deformation prior. This integration significantly enhances Gaussian density and geometry-appearance consistency. Our approach achieves real-time, high-fidelity dynamic head reconstruction at 4K resolution, with substantial improvements over baseline methods in both detail fidelity and rendering quality.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Learning with Manifolds
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Sparse volumetric reconstruction and rendering via 3D Gaussian splatting have recently enabled animatable 3D head avatars that are rendered under arbitrary viewpoints with impressive photorealism. Today, such photoreal avatars are seen as a key component in emerging applications in telepresence, extended reality, and entertainment. Building a photoreal avatar requires estimating the complex non-rigid motion of different facial components as seen in input video images; due to inaccurate motion estimation, animatable models typically present a loss of fidelity and detail when compared to their non-animatable counterparts, built from an individual facial expression. Also, recent state-of-the-art models are often affected by memory limitations that reduce the number of 3D Gaussians used for modeling, leading to lower detail and quality. To address these problems, we present a new high-detail 3D head avatar model that improves upon the state of the art, largely increasing the number of 3D Gaussians and modeling quality for rendering at 4K resolution. Our high-quality model is reconstructed from multiview input video and builds on top of a mesh-based 3D morphable model, which provides a coarse deformation layer for the head. Photoreal appearance is modelled by 3D Gaussians embedded within the continuous UVD tangent space of this mesh, allowing for more effective densification where most needed. Additionally, these Gaussians are warped by a novel UVD deformation field to capture subtle, localized motion. Our key contribution is the novel deformable Gaussian encoding and overall fitting procedure that allows our head model to preserve appearance detail, while capturing facial motion and other transient high-frequency features such as skin wrinkling.

Problem

• Improves fidelity loss in animatable 3D head avatars
• Addresses memory limitations reducing 3D Gaussian detail
• Enhances modeling quality for 4K resolution rendering

Innovation

• Texture space Gaussian avatars for high-resolution modeling
• UVD deformation field captures subtle localized motion
• Mesh-based 3D morphable model with continuous UVD space