Autoregressive Appearance Prediction for 3D Gaussian Avatars

2026-04-01

AI Summary

This work addresses the challenge of appearance discontinuities in high-fidelity 3D human avatars under novel poses, which often arise from pose-appearance ambiguities. To mitigate this issue, the authors propose a 3D Gaussian Splatting-based avatar representation integrated with a spatial MLP backbone and augmented by learnable appearance latent variables. High-quality reconstruction is achieved through joint conditioning on pose and these latent variables. During animation, appearance latents are predicted autoregressively, effectively disentangling pose from appearance and ensuring temporally smooth evolution. The method significantly enhances appearance stability and continuity under novel poses, demonstrating robust and natural rendering of complex details such as clothing, hair, and subtle facial expressions.

Abstract

A photorealistic and immersive human avatar experience demands capturing fine, person-specific details such as cloth and hair dynamics, subtle facial expressions, and characteristic motion patterns. Achieving this requires large, high-quality datasets, which often introduce ambiguities and spurious correlations when very similar poses correspond to different appearances. Models that fit these details during training can overfit and produce unstable, abrupt appearance changes for novel poses. We propose a 3D Gaussian Splatting avatar model with a spatial MLP backbone that is conditioned on both pose and an appearance latent. The latent is learned during training by an encoder, yielding a compact representation that improves reconstruction quality and helps disambiguate pose-driven renderings. At driving time, our predictor autoregressively infers the latent, producing temporally smooth appearance evolution and improved stability. Overall, our method delivers a robust and practical path to high-fidelity, stable avatar driving.

Problem

• 3D Gaussian Avatars
• appearance prediction
• pose-ambiguity
• overfitting
• temporal stability

Innovation

• 3D Gaussian Splatting
• autoregressive prediction
• appearance latent
• avatar animation
• spatial MLP