Compression of 3D Gaussian Splatting with Optimized Feature Planes and Standard Video Codecs

2025-01-06

AI Summary

To address the high storage and transmission overhead of 3D Gaussian Splatting (3DGS), which hinders practical deployment, this paper proposes a compact representation and compression framework tailored for standard video codecs. The method maps 3DGS into a video-encodable 2D tri-plane feature representation—introducing, for the first time, a progressive tri-plane structure that preserves both geometric and appearance details. Building upon this representation, we design a frequency-domain entropy model and a channel-wise adaptive bit allocation strategy, enabling end-to-end rate-distortion optimization over non-differentiable H.264/HEVC encoders. Experimental results demonstrate that our approach achieves high-fidelity rendering while significantly reducing storage and bandwidth requirements. It outperforms existing state-of-the-art methods in rate-distortion performance across multiple benchmark scenes, establishing a new trade-off frontier between compression efficiency and visual quality for neural 3D scene representations.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

3D Gaussian Splatting is a recognized method for 3D scene representation, known for its high rendering quality and speed. However, its substantial data requirements present challenges for practical applications. In this paper, we introduce an efficient compression technique that significantly reduces storage overhead by using compact representation. We propose a unified architecture that combines point cloud data and feature planes through a progressive tri-plane structure. Our method utilizes 2D feature planes, enabling continuous spatial representation. To further optimize these representations, we incorporate entropy modeling in the frequency domain, specifically designed for standard video codecs. We also propose channel-wise bit allocation to achieve a better trade-off between bitrate consumption and feature plane representation. Consequently, our model effectively leverages spatial correlations within the feature planes to enhance rate-distortion performance using standard, non-differentiable video codecs. Experimental results demonstrate that our method outperforms existing methods in data compactness while maintaining high rendering quality. Our project page is available at https://fraunhoferhhi.github.io/CodecGS

Problem

• 3D Graphics Data Compression
• Visual Quality Retention
• Storage and Bandwidth Reduction

Innovation

• 3D graphics compression
• point cloud and feature plane integration
• video codec optimization