Enhancing 3D Gaussian Splatting Compression via Spatial Condition-based Prediction

2025-03-30

AI Summary

To address the high storage and transmission overheads of 3D Gaussian Splatting (3DGS) in novel-view synthesis—stemming from its massive number of Gaussian parameters—this paper proposes the first spatially conditioned predictive compression framework. Methodologically, we leverage a scene voxel grid as a spatial prior to guide an anchor prediction module that generates initial Gaussian parameters; we further introduce an instance-aware hyperprior network and a structure-aware entropy model to efficiently capture fine-grained residuals and enable adaptive entropy coding. Key contributions include: (i) a novel spatially conditioned anchor prediction mechanism; (ii) joint instance- and structure-aware entropy modeling; and (iii) an end-to-end trainable compression architecture. Experiments demonstrate that our method achieves an average bitrate reduction of 24.42% over state-of-the-art approaches, significantly enhancing the lightweight deployment of 3DGS for real-time rendering under bandwidth-constrained scenarios.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Recently, 3D Gaussian Spatting (3DGS) has gained widespread attention in Novel View Synthesis (NVS) due to the remarkable real-time rendering performance. However, the substantial cost of storage and transmission of vanilla 3DGS hinders its further application (hundreds of megabytes or even gigabytes for a single scene). Motivated by the achievements of prediction in video compression, we introduce the prediction technique into the anchor-based Gaussian representation to effectively reduce the bit rate. Specifically, we propose a spatial condition-based prediction module to utilize the grid-captured scene information for prediction, with a residual compensation strategy designed to learn the missing fine-grained information. Besides, to further compress the residual, we propose an instance-aware hyper prior, developing a structure-aware and instance-aware entropy model. Extensive experiments demonstrate the effectiveness of our prediction-based compression framework and each technical component. Even compared with SOTA compression method, our framework still achieves a bit rate savings of 24.42 percent. Code is to be released!

Problem

• Reducing storage and transmission costs of 3D Gaussian Splatting
• Introducing prediction technique to lower bit rate
• Compressing residual with instance-aware hyper prior

Innovation

• Spatial condition-based prediction module
• Residual compensation strategy
• Instance-aware hyper prior