1000+ FPS 4D Gaussian Splatting for Dynamic Scene Rendering

2025-03-20

AI Summary

4D Gaussian Splatting (4DGS) suffers from prohibitive memory overhead and slow rendering in dynamic scene reconstruction: transient Gaussians introduce redundancy, while rasterization fails to distinguish inter-frame contributions, causing substantial redundant computation. To address this, we propose a spatiotemporal variation scoring criterion for Gaussian pruning—enabling long-sequence, low-redundancy Gaussian modeling for the first time. We further design a cross-frame dynamic activation mask that selectively rasterizes only Gaussians contributing to the current frame. Integrated with GPU-parallelized rasterization optimizations, our method achieves real-time rendering (>1000 FPS) without sacrificing visual fidelity: memory consumption is reduced by 41× and rasterization speed is accelerated by 9× compared to baseline 4DGS.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Security and Privacy: Data transparency and provenance

Abstract

4D Gaussian Splatting (4DGS) has recently gained considerable attention as a method for reconstructing dynamic scenes. Despite achieving superior quality, 4DGS typically requires substantial storage and suffers from slow rendering speed. In this work, we delve into these issues and identify two key sources of temporal redundancy. (Q1) extbf{Short-Lifespan Gaussians}: 4DGS uses a large portion of Gaussians with short temporal span to represent scene dynamics, leading to an excessive number of Gaussians. (Q2) extbf{Inactive Gaussians}: When rendering, only a small subset of Gaussians contributes to each frame. Despite this, all Gaussians are processed during rasterization, resulting in redundant computation overhead. To address these redundancies, we present extbf{4DGS-1K}, which runs at over 1000 FPS on modern GPUs. For Q1, we introduce the Spatial-Temporal Variation Score, a new pruning criterion that effectively removes short-lifespan Gaussians while encouraging 4DGS to capture scene dynamics using Gaussians with longer temporal spans. For Q2, we store a mask for active Gaussians across consecutive frames, significantly reducing redundant computations in rendering. Compared to vanilla 4DGS, our method achieves a $41 imes$ reduction in storage and $9 imes$ faster rasterization speed on complex dynamic scenes, while maintaining comparable visual quality. Please see our project page at https://4DGS-1K.github.io.

Problem

• Reduces storage and speeds up 4D Gaussian Splatting rendering.
• Addresses redundancy from short-lifespan and inactive Gaussians.
• Achieves over 1000 FPS with improved efficiency and quality.

Innovation

• Spatial-Temporal Variation Score for pruning
• Mask for active Gaussians reduction
• 1000+ FPS on modern GPUs