SparseSplat: Towards Applicable Feed-Forward 3D Gaussian Splatting with Pixel-Unaligned Prediction

2026-04-03

AI Summary

Existing feed-forward 3D Gaussian splatting methods produce spatially uniform and redundant Gaussian distributions, which struggle to meet the demands of efficient 3D reconstruction. This work proposes SparseSplat—the first feed-forward model capable of adaptively adjusting Gaussian density according to scene geometry and local information richness. By integrating an entropy-based probabilistic sampling strategy, sparse Gaussian modeling, a tailored point cloud codec network, and a pixel misalignment-aware prediction mechanism, SparseSplat effectively bridges the receptive field gap between general optimization pipelines and feed-forward architectures. Experiments demonstrate that SparseSplat achieves state-of-the-art rendering quality using only 22% of the Gaussians required by existing methods, and maintains reasonable reconstruction fidelity even when the Gaussian count is reduced to 1.5%, substantially improving representation compactness and computational efficiency.

Abstract

Recent progress in feed-forward 3D Gaussian Splatting (3DGS) has notably improved rendering quality. However, the spatially uniform and highly redundant 3DGS map generated by previous feed-forward 3DGS methods limits their integration into downstream reconstruction tasks. We propose SparseSplat, the first feed-forward 3DGS model that adaptively adjusts Gaussian density according to scene structure and information richness of local regions, yielding highly compact 3DGS maps. To achieve this, we propose entropy-based probabilistic sampling, generating large, sparse Gaussians in textureless areas and assigning small, dense Gaussians to regions with rich information. Additionally, we designed a specialized point cloud network that efficiently encodes local context and decodes it into 3DGS attributes, addressing the receptive field mismatch between the general 3DGS optimization pipeline and feed-forward models. Extensive experimental results demonstrate that SparseSplat can achieve state-of-the-art rendering quality with only 22% of the Gaussians and maintain reasonable rendering quality with only 1.5% of the Gaussians. Project page: https://victkk.github.io/SparseSplat-page/.

Problem

• 3D Gaussian Splatting
• feed-forward model
• sparse representation
• redundancy
• downstream reconstruction

Innovation

• Sparse Gaussian Splatting
• Feed-Forward 3D Reconstruction
• Entropy-Based Sampling
• Adaptive Density Control
• Point Cloud Network