Animated 3DGS Avatars in Diverse Scenes with Consistent Lighting and Shadows

2026-01-04

arXiv.org

AI Summary

This work addresses the challenge of achieving lighting consistency and realistic shadows for dynamic 3D Gaussian Splatting (3DGS) avatars interacting with complex environments. We propose Deep Gaussian Shadow Maps (DGSM), the first method to enable closed-form ray accumulation and real-time volumetric shadow computation within a purely volumetric 3DGS representation, without requiring meshing or offline optimization. By integrating spherical harmonics (SH) environment lighting probes, our approach supports efficient relighting while maintaining full compatibility with dynamic scenes. Key technical components include concentric radial shell transmittance tables, octahedral atlas sampling, and per-Gaussian fast radiative transfer. Experiments on ScanNet++, DL3DV, and SuperSplat demonstrate successful synthesis of AvatarX and ActorsHQ characters with consistent inter-character shadows and environment illumination, as well as physically plausible interactions with dynamic objects.

Technology Category

• Computer Vision: 3D Computer Vision
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Intelligent Robots: Embodied AI

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Agentic search

Abstract

We present a method for consistent lighting and shadows when animated 3D Gaussian Splatting (3DGS) avatars interact with 3DGS scenes or with dynamic objects inserted into otherwise static scenes. Our key contribution is Deep Gaussian Shadow Maps (DGSM), a modern analogue of the classical shadow mapping algorithm tailored to the volumetric 3DGS representation. Building on the classic deep shadow mapping idea, we show that 3DGS admits closed form light accumulation along light rays, enabling volumetric shadow computation without meshing. For each estimated light, we tabulate transmittance over concentric radial shells and store them in octahedral atlases, which modern GPUs can sample in real time per query to attenuate affected scene Gaussians and thus cast and receive shadows consistently. To relight moving avatars, we approximate the local environment illumination with HDRI probes represented in a spherical harmonic (SH) basis and apply a fast per Gaussian radiance transfer, avoiding explicit BRDF estimation or offline optimization. We demonstrate environment consistent lighting for avatars from AvatarX and ActorsHQ, composited into ScanNet++, DL3DV, and SuperSplat scenes, and show interactions with inserted objects. Across single and multi avatar settings, DGSM and SH relighting operate fully in the volumetric 3DGS representation, yielding coherent shadows and relighting while avoiding meshing.

Problem

• 3D Gaussian Splatting
• consistent lighting
• shadows
• avatar animation
• volumetric rendering

Innovation

• 3D Gaussian Splatting
• Deep Gaussian Shadow Maps
• Volumetric Shadows
• Spherical Harmonics Relighting
• Real-time Rendering