SalientGS: Unified SfM-to-3DGS with Importance-Guided MCMC Gaussian Allocation

2026-07-13

AI Summary

This work addresses the limitations of unordered image-based 3D reconstruction, which typically relies on costly Structure-from-Motion (SfM) preprocessing and assumes fixed camera pose interfaces. The authors propose an end-to-end differentiable framework that unifies SfM and 3D Gaussian Splatting (3DGS) reconstruction. By aggregating multi-view residuals into underfitting and redundancy signals for each Gaussian, they construct a smooth importance-weighted distribution and introduce an importance-guided Markov Chain Monte Carlo (MCMC) mechanism. This enables dynamic reallocation of Gaussian resources toward regions with higher reconstruction error, while preserving the stochastic gradient Langevin dynamics (SGLD) optimization process. The method eliminates the need for conventional SfM, achieves high-quality reconstructions within 15 minutes, and attains state-of-the-art perceptual quality.

Abstract

Reconstructing 3D scenes from unordered images remains bottlenecked by expensive Structure-from-Motion (SfM) preprocessing and frozen pose interfaces. We present SalientGS, a unified SfM-to-3D Gaussian Splatting (3DGS) pipeline. Its central contribution is importance-guided Markov Chain Monte Carlo (MCMC) Gaussian allocation, which aggregates multi-view residuals into per-Gaussian underfit and redundancy signals. These signals define a smooth importance-weighted sampling distribution that biases both birth and relocation toward underfit regions. This reallocates capacity from well-fit areas without altering the underlying stochastic gradient Langevin dynamics (SGLD). SalientGS achieves end-to-end reconstruction in 15 minutes with state-of-the-art perceptual quality. The supplementary material provides dedicated sections for Per-Scene Qualitative Comparisons and Per-Image Learned Perceptual Image Patch Similarity (LPIPS) Analysis, including failure cases. Code and evaluation scripts are available at https://github.com/Six-Bit-TX/SalientGS.

Problem

• 3D reconstruction
• Structure-from-Motion
• Gaussian Splatting
• unordered images
• pose estimation

Innovation

• importance-guided MCMC
• Gaussian allocation
• unified SfM-to-3DGS
• underfit-aware sampling
• stochastic gradient Langevin dynamics