Fast Feedforward 3D Gaussian Splatting Compression

2024-10-10

arXiv.org

AI Summary

3D Gaussian Splatting (3DGS) suffers from substantial storage overhead due to redundant Gaussian parameters, and existing compression methods require time-consuming per-scene optimization. Method: We propose the first optimization-free, single-pass feedforward real-time compression framework for 3DGS. It features a multi-path entropy coding module to jointly optimize the rate-distortion trade-off; a hierarchical context model capturing both inter-Gaussian and intra-Gaussian structural correlations; and adaptive attribute quantization coupled with entropy-constrained training. Results: Our method achieves over 20× compression ratio while matching the PSNR and SSIM of state-of-the-art per-scene optimized approaches. Compression latency drops from minutes to seconds, enabling the first high-fidelity, low-latency, and general-purpose feedforward compression of 3DGS.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Learning on the Edge & Model Compression
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

With 3D Gaussian Splatting (3DGS) advancing real-time and high-fidelity rendering for novel view synthesis, storage requirements pose challenges for their widespread adoption. Although various compression techniques have been proposed, previous art suffers from a common limitation: for any existing 3DGS, per-scene optimization is needed to achieve compression, making the compression sluggish and slow. To address this issue, we introduce Fast Compression of 3D Gaussian Splatting (FCGS), an optimization-free model that can compress 3DGS representations rapidly in a single feed-forward pass, which significantly reduces compression time from minutes to seconds. To enhance compression efficiency, we propose a multi-path entropy module that assigns Gaussian attributes to different entropy constraint paths for balance between size and fidelity. We also carefully design both inter- and intra-Gaussian context models to remove redundancies among the unstructured Gaussian blobs. Overall, FCGS achieves a compression ratio of over 20X while maintaining fidelity, surpassing most per-scene SOTA optimization-based methods. Our code is available at: https://github.com/YihangChen-ee/FCGS.

Problem

• Addresses high storage needs in 3D Gaussian Splatting for novel view synthesis.
• Eliminates per-scene optimization for faster 3DGS compression.
• Enhances compression efficiency with multi-path entropy and context models.

Innovation

• Optimization-free 3D Gaussian Splatting compression
• Multi-path entropy module for size-fidelity balance
• Inter- and intra-Gaussian context models for redundancy removal