IDESplat: Iterative Depth Probability Estimation for Generalizable 3D Gaussian Splatting

2026-01-07

arXiv.org

AI Summary

Existing 3D Gaussian splatting methods rely on single-pass depth probability estimation, which struggles to effectively integrate multi-view geometric information, resulting in coarse and unstable depth maps. This work proposes a Depth Probability Boosting Unit (DPBU) within an iterative depth estimation framework that leverages cascaded warping, epipolar attention, and multiplicative fusion strategies to enable multi-stage optimization of depth probabilities, substantially improving the accuracy of Gaussian mean predictions. The method achieves real-time performance and state-of-the-art reconstruction quality on RealEstate10K, ACID, and DL3DV benchmarks, surpassing DepthSplat by 0.33 dB in PSNR on RE10K with only 10.7% of its parameters, and demonstrates cross-dataset PSNR gains of up to 2.95 dB.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Generalizable 3D Gaussian Splatting aims to directly predict Gaussian parameters using a feed-forward network for scene reconstruction. Among these parameters, Gaussian means are particularly difficult to predict, so depth is usually estimated first and then unprojected to obtain the Gaussian sphere centers. Existing methods typically rely solely on a single warp to estimate depth probability, which hinders their ability to fully leverage cross-view geometric cues, resulting in unstable and coarse depth maps. To address this limitation, we propose IDESplat, which iteratively applies warp operations to boost depth probability estimation for accurate Gaussian mean prediction. First, to eliminate the inherent instability of a single warp, we introduce a Depth Probability Boosting Unit (DPBU) that integrates epipolar attention maps produced by cascading warp operations in a multiplicative manner. Next, we construct an iterative depth estimation process by stacking multiple DPBUs, progressively identifying potential depth candidates with high likelihood. As IDESplat iteratively boosts depth probability estimates and updates the depth candidates, the depth map is gradually refined, resulting in accurate Gaussian means. We conduct experiments on RealEstate10K, ACID, and DL3DV. IDESplat achieves outstanding reconstruction quality and state-of-the-art performance with real-time efficiency. On RE10K, it outperforms DepthSplat by 0.33 dB in PSNR, using only 10.7% of the parameters and 70% of the memory. Additionally, our IDESplat improves PSNR by 2.95 dB over DepthSplat on the DTU dataset in cross-dataset experiments, demonstrating its strong generalization ability.

Problem

• Generalizable 3D Gaussian Splatting
• depth probability estimation
• cross-view geometric cues
• Gaussian mean prediction
• depth map refinement

Innovation

• Iterative Depth Estimation
• Depth Probability Boosting
• 3D Gaussian Splatting
• Epipolar Attention
• Generalizable Reconstruction