PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting

2024-06-14

arXiv.org

AI Summary

To address the storage and memory bottlenecks caused by the excessive number of Gaussians in 3D Gaussian Splatting (3D-GS), this paper proposes an uncertainty-aware pruning method grounded in second-order sensitivity analysis of reconstruction error. Unlike heuristic pruning strategies, our approach establishes the first principled pruning scoring mechanism: it quantifies each Gaussian’s contribution via second-order Taylor approximation and spatial parameter sensitivity analysis. Pruning is performed iteratively, interleaved with rendering-based fine-tuning to enhance compression robustness. Under a 90% Gaussian pruning ratio, our method achieves a 3.56× rendering speedup while outperforming state-of-the-art methods across all major metrics—PSNR, SSIM, and LPIPS. Extensive evaluation on the Mip-NeRF 360, Tanks & Temples, and Deep Blending benchmarks demonstrates that our method preserves high visual fidelity and foreground detail even at extreme compression ratios.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Recent advances in novel view synthesis have enabled real-time rendering speeds with high reconstruction accuracy. 3D Gaussian Splatting (3D-GS), a foundational point-based parametric 3D scene representation, models scenes as large sets of 3D Gaussians. However, complex scenes can consist of millions of Gaussians, resulting in high storage and memory requirements that limit the viability of 3D-GS on devices with limited resources. Current techniques for compressing these pretrained models by pruning Gaussians rely on combining heuristics to determine which Gaussians to remove. At high compression ratios, these pruned scenes suffer from heavy degradation of visual fidelity and loss of foreground details. In this paper, we propose a principled sensitivity pruning score that preserves visual fidelity and foreground details at significantly higher compression ratios than existing approaches. It is computed as a second-order approximation of the reconstruction error on the training views with respect to the spatial parameters of each Gaussian. Additionally, we propose a multi-round prune-refine pipeline that can be applied to any pretrained 3D-GS model without changing its training pipeline. After pruning 90% of Gaussians, a substantially higher percentage than previous methods, our PUP 3D-GS pipeline increases average rendering speed by 3.56$ imes$ while retaining more salient foreground information and achieving higher image quality metrics than existing techniques on scenes from the Mip-NeRF 360, Tanks&Temples, and Deep Blending datasets.

Problem

• Reducing storage and memory in 3D Gaussian Splatting models
• Preserving visual fidelity at high compression ratios
• Improving rendering speed without losing foreground details

Innovation

• Principled sensitivity pruning score for 3D-GS
• Multi-round prune-refine pipeline for pretrained models
• Higher compression ratios with retained visual fidelity