S2Gaussian: Sparse-View Super-Resolution 3D Gaussian Splatting

2025-03-06

AI Summary

This work addresses the challenging problem of high-fidelity 3D reconstruction from sparse, low-resolution multi-view images—characterized by both insufficient viewpoint coverage and limited per-view spatial resolution. To this end, we propose the first two-stage sparse super-resolution Gaussian splatting framework. Our method introduces several key innovations: Gaussian Shuffle-Split initialization, depth-regularized Gaussian optimization, pseudo-view augmentation, multi-view inconsistency modeling, and a 3D-robust gradient update strategy—enabling end-to-end joint geometric-appearance optimization. Evaluated on standard benchmarks including ScanNet and Tanks & Temples, our approach achieves new state-of-the-art performance, significantly improving geometric consistency and texture fidelity. Notably, it reconstructs high-fidelity 3D scenes from as few as 2–4 low-resolution input views, demonstrating strong robustness to extreme sparsity and resolution degradation.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

In this paper, we aim ambitiously for a realistic yet challenging problem, namely, how to reconstruct high-quality 3D scenes from sparse low-resolution views that simultaneously suffer from deficient perspectives and clarity. Whereas existing methods only deal with either sparse views or low-resolution observations, they fail to handle such hybrid and complicated scenarios. To this end, we propose a novel Sparse-view Super-resolution 3D Gaussian Splatting framework, dubbed S2Gaussian, that can reconstruct structure-accurate and detail-faithful 3D scenes with only sparse and low-resolution views. The S2Gaussian operates in a two-stage fashion. In the first stage, we initially optimize a low-resolution Gaussian representation with depth regularization and densify it to initialize the high-resolution Gaussians through a tailored Gaussian Shuffle Split operation. In the second stage, we refine the high-resolution Gaussians with the super-resolved images generated from both original sparse views and pseudo-views rendered by the low-resolution Gaussians. In which a customized blur-free inconsistency modeling scheme and a 3D robust optimization strategy are elaborately designed to mitigate multi-view inconsistency and eliminate erroneous updates caused by imperfect supervision. Extensive experiments demonstrate superior results and in particular establishing new state-of-the-art performances with more consistent geometry and finer details.

Problem

• Reconstruct high-quality 3D scenes from sparse low-resolution views.
• Address hybrid scenarios of sparse views and low-resolution observations.
• Mitigate multi-view inconsistency and eliminate erroneous updates.

Innovation

• Two-stage sparse-view super-resolution 3D reconstruction
• Gaussian Shuffle Split for high-resolution initialization
• Blur-free inconsistency modeling for robust optimization