Image-Conditioned 3D Gaussian Splat Quantization

2025-08-21

AI Summary

Existing 3D Gaussian Splatting (3DGS) compression methods struggle to simultaneously achieve high compression ratios and long-term updateability: they are limited to MB-scale compression and lack support for dynamic scene updates after archival. To address this, we propose an image-conditioned quantization framework that jointly models spatial and attribute correlations among Gaussians, enables cross-scene codebook sharing, and performs end-to-end joint optimization of encoding, quantization, and decoding. Furthermore, we introduce an image-guided adaptive decoding mechanism to refine reconstructions. Our method achieves KB-scale compression—reducing storage by one to two orders of magnitude over state-of-the-art approaches—while supporting on-demand post-archival updates. Extensive experiments on large-scale scenes demonstrate significant improvements in both compression efficiency and rendering quality. The code and models will be publicly released.

Technology Category

• Computer Vision: 3D Computer Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Learning on the Edge & Model Compression

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

3D Gaussian Splatting (3DGS) has attracted considerable attention for enabling high-quality real-time rendering. Although 3DGS compression methods have been proposed for deployment on storage-constrained devices, two limitations hinder archival use: (1) they compress medium-scale scenes only to the megabyte range, which remains impractical for large-scale scenes or extensive scene collections; and (2) they lack mechanisms to accommodate scene changes after long-term archival. To address these limitations, we propose an Image-Conditioned Gaussian Splat Quantizer (ICGS-Quantizer) that substantially enhances compression efficiency and provides adaptability to scene changes after archiving. ICGS-Quantizer improves quantization efficiency by jointly exploiting inter-Gaussian and inter-attribute correlations and by using shared codebooks across all training scenes, which are then fixed and applied to previously unseen test scenes, eliminating the overhead of per-scene codebooks. This approach effectively reduces the storage requirements for 3DGS to the kilobyte range while preserving visual fidelity. To enable adaptability to post-archival scene changes, ICGS-Quantizer conditions scene decoding on images captured at decoding time. The encoding, quantization, and decoding processes are trained jointly, ensuring that the codes, which are quantized representations of the scene, are effective for conditional decoding. We evaluate ICGS-Quantizer on 3D scene compression and 3D scene updating. Experimental results show that ICGS-Quantizer consistently outperforms state-of-the-art methods in compression efficiency and adaptability to scene changes. Our code, model, and data will be publicly available on GitHub.

Problem

• Compressing 3D Gaussian Splatting scenes to kilobyte range for storage
• Enabling adaptability to scene changes after long-term archival
• Eliminating per-scene codebook overhead through shared codebooks

Innovation

• Image-conditioned quantization for 3D Gaussian splatting
• Shared codebooks across scenes to reduce storage
• Joint encoding and decoding for scene adaptability