StereoSplat+: Feed-Forward Stereo Gaussian Splatting with Diffusion-Assisted Progressive Inference

2026-07-09

AI Summary

This work addresses the challenge of reconstructing high-quality 3D Gaussian Splatting (3DGS) scenes from a single stereo image pair—a task severely hindered by occlusions, limited field of view, and geometric ambiguity, particularly in causal settings such as robotics and augmented reality. We propose StereoSplat+, a diffusion-enhanced, feed-forward stereo Gaussian splatting framework that integrates cost volumes and triplane features through a novel StereoSplat module and employs a single-step diffusion model to drive a progressive inference mechanism, iteratively refining novel-view synthesis to enhance the 3D Gaussian representation. Our method achieves, for the first time, causal, feed-forward high-fidelity 3DGS reconstruction using only a single stereo pair, and introduces a continuous pose encoding that generalizes across arbitrary numbers of views and camera configurations. Evaluated on KITTI-360, StereoSplat+ significantly improves rendering quality and geometric accuracy, especially in occluded regions and under strong viewpoint extrapolation, outperforming existing feed-forward approaches.

Abstract

Recent advances in 3D Gaussian Splatting (3DGS) have enabled high-quality, render-ready scene representations for novel-view synthesis. However, most existing 3DGS pipelines rely on multi-view observations (or non-causal access to future frames) to achieve sufficient coverage, which is often unavailable in on-device robotics and AR settings where sensing is restricted to a single stereo rig. Recovering a high-quality 3DGS scene from one stereo observation, therefore, remains challenging due to occlusions, limited field of view, and missing geometry. We present StereoSplat+, a diffusion-enhanced feed-forward framework that enables causal reconstruction from a single stereo pair. Our method builds on two key components. First, we propose StereoSplat, an input-invariant feed-forward 3D Gaussian estimator that takes a variable number of posed stereo pairs as input and predicts high-quality 3D Gaussians. StereoSplat fuses complementary geometry cues via a cost-volume branch and a triplane-based 3D volume branch and leverages continuous pose encoding to generalize across view counts and camera configurations. Second, since multiple posed stereo pairs are typically unavailable at inference time, we introduce a diffusion-enhanced one-shot progressive inference scheme called StereoSplat+: starting from one stereo pair, we render novel stereo views from the predicted 3DGS, refine them with a one-step diffusion enhancer, and feed them back as additional inputs to update the 3DGS. Experiments on the KITTI-360 dataset show that StereoSplat+ improves novel-view rendering quality and geometry accuracy, especially in occluded regions and under strong view extrapolation, outperforming recent feed-forward 3DGS baselines.

Problem

• 3D Gaussian Splatting
• stereo vision
• single-view reconstruction
• occlusion
• novel-view synthesis

Innovation

• 3D Gaussian Splatting
• stereo reconstruction
• diffusion enhancement
• feed-forward inference
• novel-view synthesis