SA-ResGS: Self-Augmented Residual 3D Gaussian Splatting for Next Best View Selection

2026-01-06

arXiv.org

AI Summary

This work addresses the challenges of unreliable uncertainty estimation and insufficient supervision for Gaussian points in active scene reconstruction under sparse, wide-baseline viewpoints. To this end, we propose a physics-guided next-best-view (NBV) selection framework that integrates self-augmented point cloud generation with an uncertainty-aware residual learning mechanism tailored for 3D Gaussian splatting. By incorporating triangulation constraints, uncertainty-driven filtering, and hard negative mining, our approach enables effective supervision and debiased quantification of weakly contributing Gaussian points. The method significantly improves scene coverage efficiency, reconstruction fidelity, and robustness in viewpoint selection, outperforming existing approaches in active reconstruction tasks.

Technology Category

• Machine Learning: Active Learning
• Reasoning under Uncertainty: Uncertainty Representations
• Computer Vision: Scene Analysis & Understanding

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

We propose Self-Augmented Residual 3D Gaussian Splatting (SA-ResGS), a novel framework to stabilize uncertainty quantification and enhancing uncertainty-aware supervision in next-best-view (NBV) selection for active scene reconstruction. SA-ResGS improves both the reliability of uncertainty estimates and their effectiveness for supervision by generating Self-Augmented point clouds (SA-Points) via triangulation between a training view and a rasterized extrapolated view, enabling efficient scene coverage estimation. While improving scene coverage through physically guided view selection, SA-ResGS also addresses the challenge of under-supervised Gaussians, exacerbated by sparse and wide-baseline views, by introducing the first residual learning strategy tailored for 3D Gaussian Splatting. This targeted supervision enhances gradient flow in high-uncertainty Gaussians by combining uncertainty-driven filtering with dropout- and hard-negative-mining-inspired sampling. Our contributions are threefold: (1) a physically grounded view selection strategy that promotes efficient and uniform scene coverage; (2) an uncertainty-aware residual supervision scheme that amplifies learning signals for weakly contributing Gaussians, improving training stability and uncertainty estimation across scenes with diverse camera distributions; (3) an implicit unbiasing of uncertainty quantification as a consequence of constrained view selection and residual supervision, which together mitigate conflicting effects of wide-baseline exploration and sparse-view ambiguity in NBV planning. Experiments on active view selection demonstrate that SA-ResGS outperforms state-of-the-art baselines in both reconstruction quality and view selection robustness.

Problem

• Next Best View Selection
• Uncertainty Quantification
• 3D Gaussian Splatting
• Active Scene Reconstruction
• Sparse Views

Innovation

• Self-Augmented
• Residual Learning
• 3D Gaussian Splatting
• Uncertainty-Aware Supervision
• Next-Best-View Selection