Topology-Aware Optimization of Gaussian Primitives for Human-Centric Volumetric Videos

2025-09-09

AI Summary

Long-term tracking and topological change modeling in dynamic scenes remain critical challenges for immersive volumetric video rendering. This paper proposes TaoGS, a topology-aware dynamic Gaussian splatting framework that decouples motion and appearance representation. Its core innovation is the first introduction of a topology-aware mechanism: sparse motion Gaussians model structural evolution, while lifespan-aware dynamic appearance Gaussians reconstruct fine-grained textures. A global Gaussian lookup table enables seamless integration with video encoders, achieving up to 40× compression. Motion Gaussians are jointly optimized via spatiotemporal tracking and photometric constraints, and appearance Gaussians are initialized via non-rigid warping. Evaluated on challenging topology-varying scenarios—such as clothing changes—TaoGS significantly improves long-term temporal consistency and rendering fidelity, accelerates training, and enables high-fidelity six-degree-of-freedom immersive reconstruction.

Technology Category

• Computer Vision: Motion & Tracking
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Volumetric video is emerging as a key medium for digitizing the dynamic physical world, creating the virtual environments with six degrees of freedom to deliver immersive user experiences. However, robustly modeling general dynamic scenes, especially those involving topological changes while maintaining long-term tracking remains a fundamental challenge. In this paper, we present TaoGS, a novel topology-aware dynamic Gaussian representation that disentangles motion and appearance to support, both, long-range tracking and topological adaptation. We represent scene motion with a sparse set of motion Gaussians, which are continuously updated by a spatio-temporal tracker and photometric cues that detect structural variations across frames. To capture fine-grained texture, each motion Gaussian anchors and dynamically activates a set of local appearance Gaussians, which are non-rigidly warped to the current frame to provide strong initialization and significantly reduce training time. This activation mechanism enables efficient modeling of detailed textures and maintains temporal coherence, allowing high-fidelity rendering even under challenging scenarios such as changing clothes. To enable seamless integration into codec-based volumetric formats, we introduce a global Gaussian Lookup Table that records the lifespan of each Gaussian and organizes attributes into a lifespan-aware 2D layout. This structure aligns naturally with standard video codecs and supports up to 40 compression. TaoGS provides a unified, adaptive solution for scalable volumetric video under topological variation, capturing moments where "elegance in motion" and "Power in Stillness", delivering immersive experiences that harmonize with the physical world.

Problem

• Modeling dynamic scenes with topological changes while maintaining long-term tracking
• Achieving high-fidelity rendering under challenging scenarios like clothing changes
• Enabling efficient compression and integration into standard volumetric video formats

Innovation

• Topology-aware dynamic Gaussian representation disentangles motion and appearance
• Sparse motion Gaussians updated by spatio-temporal tracker with photometric cues
• Global Gaussian Lookup Table enables codec integration with lifespan-aware compression