Sparse-View 3D Gaussian Splatting in the Wild

2026-04-30

AI Summary

This work addresses the challenge of high-quality novel view synthesis from sparse-view images captured under unconstrained real-world conditions by proposing an innovative framework based on 3D Gaussian splatting. The method effectively handles unconstrained sparse inputs through three key components: reference-image-guided view optimization, transient-mask-guided pseudo-view generation using a diffusion model, and a sparsity-aware Gaussian replication mechanism. These components collectively enhance geometric and appearance modeling in sparsely observed regions. Evaluated on public benchmarks, the proposed approach significantly outperforms existing state-of-the-art methods, achieving a 17.2% improvement in PSNR, a 10.8% gain in SSIM, and a 4.0% reduction in LPIPS, thereby enabling high-fidelity 3D rendering from limited input views.

Abstract

We propose a 3D novel sparse-view synthesis framework for unconstrained real-world scenarios that contain distractors. Unlike existing methods that primarily perform novel-view synthesis from a sparse set of constrained images without transient elements or leverage unconstrained dense image collections to enhance 3D representation in real-world scenarios, our method not only effectively tackles sparse unconstrained image collections, but also shows high-quality 3D rendering results. To do this, we introduce reference-guided view refinement with a diffusion model using a transient mask and a reference image to enhance the 3D representation and mitigate artifacts in rendered views. Furthermore, we address sparse regions in the Gaussian field via pseudo-view generation along with a sparsity-aware Gaussian replication strategy to amplify Gaussians in the sparse regions. Extensive experiments on publicly available datasets demonstrate that our methodology consistently outperforms existing methods (e.g., PSNR - 17.2%, SSIM - 10.8%, LPIPS - 4.0%) and provides high-fidelity 3D rendering results. This advancement paves the way for realizing unconstrained real-world scenarios without labor-intensive data acquisition. Our project page is available at $\href{https://robotic-vision-lab.github.io/SaveWildGS/}{here}$

Problem

• sparse-view synthesis
• 3D Gaussian splatting
• unconstrained real-world scenarios
• transient elements
• novel-view synthesis

Innovation

• Sparse-View Synthesis
• 3D Gaussian Splatting
• Diffusion Model
• Transient Mask
• Pseudo-View Generation