Speedy Deformable 3D Gaussian Splatting: Fast Rendering and Compression of Dynamic Scenes

2025-06-09

AI Summary

To address the high rendering latency and excessive GPU memory/computation overhead caused by per-Gaussian temporal deformation in dynamic 3D/4D Gaussian Splatting (3DGS/4DGS), this paper proposes a temporal-aware lightweight acceleration framework. Our method introduces: (1) a temporally sensitive pruning score coupled with an annealing-based smooth pruning strategy for efficient sparsification; and (2) GroupFlow, a motion clustering technique that replaces per-Gaussian deformation prediction with rigid-motion group modeling, drastically reducing neural inference frequency. Evaluated on NeRF-DS, our approach achieves 10.37× rendering speedup, 7.71× model compression, and 2.71× training acceleration. On D-NeRF and HyperNeRF-vrig, it delivers 4.20× and 58.23× speedups, respectively—while preserving rendering quality and generalization across diverse dynamic scenes.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Hardware-aware ML

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Recent extensions of 3D Gaussian Splatting (3DGS) to dynamic scenes achieve high-quality novel view synthesis by using neural networks to predict the time-varying deformation of each Gaussian. However, performing per-Gaussian neural inference at every frame poses a significant bottleneck, limiting rendering speed and increasing memory and compute requirements. In this paper, we present Speedy Deformable 3D Gaussian Splatting (SpeeDe3DGS), a general pipeline for accelerating the rendering speed of dynamic 3DGS and 4DGS representations by reducing neural inference through two complementary techniques. First, we propose a temporal sensitivity pruning score that identifies and removes Gaussians with low contribution to the dynamic scene reconstruction. We also introduce an annealing smooth pruning mechanism that improves pruning robustness in real-world scenes with imprecise camera poses. Second, we propose GroupFlow, a motion analysis technique that clusters Gaussians by trajectory similarity and predicts a single rigid transformation per group instead of separate deformations for each Gaussian. Together, our techniques accelerate rendering by $10.37 imes$, reduce model size by $7.71 imes$, and shorten training time by $2.71 imes$ on the NeRF-DS dataset. SpeeDe3DGS also improves rendering speed by $4.20 imes$ and $58.23 imes$ on the D-NeRF and HyperNeRF vrig datasets. Our methods are modular and can be integrated into any deformable 3DGS or 4DGS framework.

Problem

• Accelerate dynamic 3DGS rendering speed
• Reduce model size and compute requirements
• Improve pruning robustness in real-world scenes

Innovation

• Temporal sensitivity pruning for dynamic scenes
• GroupFlow clusters Gaussians by trajectory
• Annealing smooth pruning enhances robustness