Optimized 3D Gaussian Splatting using Coarse-to-Fine Image Frequency Modulation

2025-03-18

AI Summary

3D Gaussian Splatting (3DGS) achieves high-fidelity, real-time novel-view synthesis but suffers from prohibitive GPU memory and disk storage overheads, hindering deployment on consumer-grade hardware. To address this, we propose a frequency-modulation-driven coarse-to-fine optimization framework. Leveraging frequency-domain analysis, our method guides image modulation, coarse-grained Gaussian initialization, and fine-grained adaptive sparsification during training—significantly reducing the number of Gaussian primitives. The approach inherently supports multi-level-of-detail representations and integrates seamlessly into standard 3DGS pipelines while improving generalization. Experiments demonstrate an average 62% reduction in Gaussian primitives, a 40% decrease in peak training GPU memory consumption, a 20% speedup in optimization, and consistent improvements in PSNR and SSIM metrics—without compromising rendering quality.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Optimization

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

The field of Novel View Synthesis has been revolutionized by 3D Gaussian Splatting (3DGS), which enables high-quality scene reconstruction that can be rendered in real-time. 3DGS-based techniques typically suffer from high GPU memory and disk storage requirements which limits their practical application on consumer-grade devices. We propose Opti3DGS, a novel frequency-modulated coarse-to-fine optimization framework that aims to minimize the number of Gaussian primitives used to represent a scene, thus reducing memory and storage demands. Opti3DGS leverages image frequency modulation, initially enforcing a coarse scene representation and progressively refining it by modulating frequency details in the training images. On the baseline 3DGS, we demonstrate an average reduction of 62% in Gaussians, a 40% reduction in the training GPU memory requirements and a 20% reduction in optimization time without sacrificing the visual quality. Furthermore, we show that our method integrates seamlessly with many 3DGS-based techniques, consistently reducing the number of Gaussian primitives while maintaining, and often improving, visual quality. Additionally, Opti3DGS inherently produces a level-of-detail scene representation at no extra cost, a natural byproduct of the optimization pipeline. Results and code will be made publicly available.

Problem

• Reduces GPU memory and storage for 3D Gaussian Splatting
• Minimizes Gaussian primitives without losing visual quality
• Enables efficient real-time rendering on consumer devices

Innovation

• Coarse-to-fine optimization reduces Gaussian primitives.
• Image frequency modulation minimizes memory and storage.
• Level-of-detail scene representation improves visual quality.