SVG-Head: Hybrid Surface-Volumetric Gaussians for High-Fidelity Head Reconstruction and Real-Time Editing

2025-08-13

AI Summary

To address the limited editability and suboptimal rendering quality inherent in implicit representations for head avatar reconstruction, this paper proposes a hybrid surface-voxel Gaussian representation. Our method explicitly decouples geometry and appearance by generating high-fidelity texture maps for Gaussian avatars—the first such approach. We construct 3D Gaussians anchored to the FLAME mesh, introducing a mesh-aware Gaussian UV mapping and a hierarchical optimization strategy to accurately model non-Lambertian regions. Evaluated on the NeRSemble dataset, our method achieves high-fidelity, real-time rendering (≥30 FPS) and enables intuitive, fine-grained texture editing. Compared to state-of-the-art implicit methods, it significantly improves the joint performance of editability and visual fidelity—bridging a critical gap between controllability and photorealism in neural head avatars.

Technology Category

• Computer Vision: 3D Computer Vision
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

Creating high-fidelity and editable head avatars is a pivotal challenge in computer vision and graphics, boosting many AR/VR applications. While recent advancements have achieved photorealistic renderings and plausible animation, head editing, especially real-time appearance editing, remains challenging due to the implicit representation and entangled modeling of the geometry and global appearance. To address this, we propose Surface-Volumetric Gaussian Head Avatar (SVG-Head), a novel hybrid representation that explicitly models the geometry with 3D Gaussians bound on a FLAME mesh and leverages disentangled texture images to capture the global appearance. Technically, it contains two types of Gaussians, in which surface Gaussians explicitly model the appearance of head avatars using learnable texture images, facilitating real-time texture editing, while volumetric Gaussians enhance the reconstruction quality of non-Lambertian regions (e.g., lips and hair). To model the correspondence between 3D world and texture space, we provide a mesh-aware Gaussian UV mapping method, which leverages UV coordinates given by the FLAME mesh to obtain sharp texture images and real-time rendering speed. A hierarchical optimization strategy is further designed to pursue the optimal performance in both reconstruction quality and editing flexibility. Experiments on the NeRSemble dataset show that SVG-Head not only generates high-fidelity rendering results, but also is the first method to obtain explicit texture images for Gaussian head avatars and support real-time appearance editing.

Problem

• Hybrid representation for high-fidelity head avatar reconstruction
• Real-time appearance editing with disentangled texture images
• Mesh-aware Gaussian UV mapping for sharp textures

Innovation

• Hybrid Surface-Volumetric Gaussian representation for avatars
• Mesh-aware Gaussian UV mapping for texture sharpness
• Hierarchical optimization for quality and editing flexibility