Sparse-to-Complete: From Sparse Image Captures to Complete 3D Scenes

2026-05-07

AI Summary

This work addresses the challenge of 3D scene reconstruction from extremely sparse inputs—only six to eight images—where existing methods often suffer from missing geometry and artifacts. The authors propose S2C-3D, a novel framework that integrates scene-finetuned diffusion models with training-free, view-consistency-guided sampling, and introduces visibility-aware camera trajectory optimization. This synergistic approach enables the generation of geometrically complete and texture-faithful 3D Gaussian scenes. Under such highly sparse input conditions, S2C-3D substantially outperforms current state-of-the-art methods, achieving high-quality reconstructions free of structural gaps and visual blurriness.

Abstract

We introduce S2C-3D, a novel sparse-view 3D reconstruction framework for high-fidelity and complete scene reconstruction from as few as six to eight images. Our framework features three components: a specialized diffusion model for scene-specific image restoration, a training-free view-consistency conditioned sampling process in the diffusion model for refined Gaussian optimization, and a camera trajectory planning scheme to ensure comprehensive scene coverage. The specialized diffusion model is developed by finetuning a pretrained architecture on the input views and their corresponding degraded counterparts. The adaptation to the scene distribution allows the model to repair Gaussian renderings while effectively eliminating domain gaps. Meanwhile, the trajectory planning scheme optimizes scene coverage by connecting each newly sampled camera to its two nearest neighbors. By iteratively constructing paths and retaining only those that significantly enhance visibility, the scheme establishes a trajectory that covers the entire scene. To address multi-view conflicts, the view-consistency conditioned sampling process quantifies the consistency between neighboring repaired images. This information is injected as a condition into the sampling process of the frozen diffusion model, facilitating the generation of view-consistent images without additional training. Consequently, our approach produces high-fidelity 3D Gaussians that are robust to artifacts. Experimental results demonstrate that S2C-3D outperforms state-of-the-art methods, constructing high-quality scenes that are free from missing regions, blurring, or other artifacts with very sparse inputs. The source code and data are available at https://gapszju.github.io/S2C-3D.

Problem

• sparse-view reconstruction
• 3D scene completion
• high-fidelity reconstruction
• view consistency
• 3D Gaussian reconstruction

Innovation

• sparse-view reconstruction
• diffusion model
• view consistency
• camera trajectory planning
• 3D Gaussian