SurFhead: Affine Rig Blending for Geometrically Accurate 2D Gaussian Surfel Head Avatars

2024-10-15

arXiv.org

AI Summary

Existing Gaussian-primitive-based methods for head avatar rendering are constrained by similarity transformations, limiting their ability to model critical geometric deformations such as stretching and shearing—resulting in geometric distortion in fine-detail reconstruction and poor generalization to unseen poses. This paper proposes a novel framework integrating 2D Gaussian surfels with mesh-based deformation transfer. We introduce polar decomposition to jointly decompose affine transformations applied to surfel positions and normals, unifying geometric fidelity and controllability. Furthermore, we combine affine transformation interpolation with ray-surfel depth intersection computation to enable high-fidelity surface reconstruction and analytically differentiable normal estimation. Experiments demonstrate state-of-the-art geometric reconstruction accuracy and joint normal/image rendering quality under extreme poses, while significantly improving generalization to unseen poses and mesh reconstruction fidelity.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Machine Learning: Learning with Manifolds
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Data transparency and provenance
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Recent advancements in head avatar rendering using Gaussian primitives have achieved significantly high-fidelity results. Although precise head geometry is crucial for applications like mesh reconstruction and relighting, current methods struggle to capture intricate geometric details and render unseen poses due to their reliance on similarity transformations, which cannot handle stretch and shear transforms essential for detailed deformations of geometry. To address this, we propose SurFhead, a novel method that reconstructs riggable head geometry from RGB videos using 2D Gaussian surfels, which offer well-defined geometric properties, such as precise depth from fixed ray intersections and normals derived from their surface orientation, making them advantageous over 3D counterparts. SurFhead ensures high-fidelity rendering of both normals and images, even in extreme poses, by leveraging classical mesh-based deformation transfer and affine transformation interpolation. SurFhead introduces precise geometric deformation and blends surfels through polar decomposition of transformations, including those affecting normals. Our key contribution lies in bridging classical graphics techniques, such as mesh-based deformation, with modern Gaussian primitives, achieving state-of-the-art geometry reconstruction and rendering quality. Unlike previous avatar rendering approaches, SurFhead enables efficient reconstruction driven by Gaussian primitives while preserving high-fidelity geometry.

Problem

• Captures intricate geometric details in head avatars
• Handles stretch and shear transforms for detailed deformations
• Ensures high-fidelity rendering in extreme poses

Innovation

• Uses 2D Gaussian surfels for precise geometry
• Leverages affine transformation interpolation for deformations
• Blends surfels via polar decomposition of transformations