MoDec-GS: Global-to-Local Motion Decomposition and Temporal Interval Adjustment for Compact Dynamic 3D Gaussian Splatting

2025-01-07

AI Summary

To address the challenges of complex motion modeling, large model size, and low rendering efficiency in dynamic-scene 3D Gaussian Splatting reconstruction, this paper proposes an efficient dynamic video reconstruction framework. Our method introduces three key innovations: (1) Global-to-Local Motion Decomposition (GLMD), which jointly optimizes coarse global deformation via a Global Anchor Deformation module and fine-grained per-Gaussian deformations; (2) Time-Interval Adaptation (TIA), enhancing temporal modeling flexibility by dynamically adjusting sampling intervals; and (3) a hybrid representation integrating implicit Scaffold priors with dynamic 3D Gaussian Splatting. Evaluated on real-world dynamic videos, our approach achieves an average 70% reduction in model size compared to state-of-the-art methods, while preserving or improving rendering quality. This significantly advances dynamic scene representation fidelity and storage efficiency.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

3D Gaussian Splatting (3DGS) has made significant strides in scene representation and neural rendering, with intense efforts focused on adapting it for dynamic scenes. Despite delivering remarkable rendering quality and speed, existing methods struggle with storage demands and representing complex real-world motions. To tackle these issues, we propose MoDecGS, a memory-efficient Gaussian splatting framework designed for reconstructing novel views in challenging scenarios with complex motions. We introduce GlobaltoLocal Motion Decomposition (GLMD) to effectively capture dynamic motions in a coarsetofine manner. This approach leverages Global Canonical Scaffolds (Global CS) and Local Canonical Scaffolds (Local CS), extending static Scaffold representation to dynamic video reconstruction. For Global CS, we propose Global Anchor Deformation (GAD) to efficiently represent global dynamics along complex motions, by directly deforming the implicit Scaffold attributes which are anchor position, offset, and local context features. Next, we finely adjust local motions via the Local Gaussian Deformation (LGD) of Local CS explicitly. Additionally, we introduce Temporal Interval Adjustment (TIA) to automatically control the temporal coverage of each Local CS during training, allowing MoDecGS to find optimal interval assignments based on the specified number of temporal segments. Extensive evaluations demonstrate that MoDecGS achieves an average 70% reduction in model size over stateoftheart methods for dynamic 3D Gaussians from realworld dynamic videos while maintaining or even improving rendering quality.

Problem

• 3D Gaussian Splatting
• Dynamic Scenes
• Efficiency Issues

Innovation

• Global-to-Local Motion Decomposition
• Time Interval Adjustment
• Model Size Reduction