3DGabSplat: 3D Gabor Splatting for Frequency-adaptive Radiance Field Rendering

2025-08-07

AI Summary

3D Gaussian Splatting (3DGS) suffers from inherent limitations in representing high-frequency geometric and textural details due to the low-pass nature of isotropic Gaussian kernels, leading to primitive redundancy, suboptimal training/rendering efficiency, and excessive memory consumption. To address this, we propose 3D Gabor Splatting: a novel radiance field representation that replaces Gaussian primitives with learnable, multi-directional, multi-band 3D Gabor filter banks—enabling anisotropic frequency response. We further introduce a frequency-adaptive optimization mechanism to dynamically tune each primitive’s spectral bandwidth, and develop an efficient, differentiable CUDA rasterizer for end-to-end rendering. Our method is plug-and-play compatible with existing 3DGS pipelines. Evaluated on both synthetic and real-world datasets, it achieves up to +1.35 dB PSNR improvement while significantly reducing primitive count and GPU memory usage, enabling high-fidelity, computationally efficient novel view synthesis.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Mixed Discrete/Continuous Search
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Recent prominence in 3D Gaussian Splatting (3DGS) has enabled real-time rendering while maintaining high-fidelity novel view synthesis. However, 3DGS resorts to the Gaussian function that is low-pass by nature and is restricted in representing high-frequency details in 3D scenes. Moreover, it causes redundant primitives with degraded training and rendering efficiency and excessive memory overhead. To overcome these limitations, we propose 3D Gabor Splatting (3DGabSplat) that leverages a novel 3D Gabor-based primitive with multiple directional 3D frequency responses for radiance field representation supervised by multi-view images. The proposed 3D Gabor-based primitive forms a filter bank incorporating multiple 3D Gabor kernels at different frequencies to enhance flexibility and efficiency in capturing fine 3D details. Furthermore, to achieve novel view rendering, an efficient CUDA-based rasterizer is developed to project the multiple directional 3D frequency components characterized by 3D Gabor-based primitives onto the 2D image plane, and a frequency-adaptive mechanism is presented for adaptive joint optimization of primitives. 3DGabSplat is scalable to be a plug-and-play kernel for seamless integration into existing 3DGS paradigms to enhance both efficiency and quality of novel view synthesis. Extensive experiments demonstrate that 3DGabSplat outperforms 3DGS and its variants using alternative primitives, and achieves state-of-the-art rendering quality across both real-world and synthetic scenes. Remarkably, we achieve up to 1.35 dB PSNR gain over 3DGS with simultaneously reduced number of primitives and memory consumption.

Problem

• Overcome 3DGS limitations in high-frequency detail representation
• Reduce redundant primitives and memory overhead in rendering
• Enhance training and rendering efficiency with adaptive optimization

Innovation

• 3D Gabor-based primitive for frequency responses
• CUDA-based rasterizer for efficient rendering
• Frequency-adaptive mechanism for optimization