Bootstrap-GS: Self-Supervised Augmentation for High-Fidelity Gaussian Splatting

2024-04-29

AI Summary

3D Gaussian Splatting (3D-GS) suffers from severe reconstruction artifacts in novel-view synthesis under large viewpoint deviations and exhibits dilation and aliasing artifacts during scaling—primarily due to insufficient training-view coverage. To address this, we propose a self-supervised, bootstrapped view-sampling augmentation framework: leveraging differentiable rendering to synthesize trustworthy novel-view pseudo-ground truths, then iteratively expanding the training distribution via viewpoint consistency constraints, Gaussian parameter re-optimization, and pseudo-ground-truth distillation. Our method introduces, for the first time, a fully self-supervised, annotation-free bootstrapping mechanism for view augmentation—compatible with diverse Gaussian representations. Evaluated on multiple benchmarks, it achieves PSNR gains of 1.2–2.8 dB and SSIM scores exceeding 0.92, while significantly suppressing artifacts and improving geometric fidelity and generalization to unseen viewpoints.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Unsupervised & Self-Supervised Learning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recent advancements in 3D Gaussian Splatting (3D-GS) have established new benchmarks for rendering quality and efficiency in 3D reconstruction. However, 3D-GS faces critical limitations when generating novel views that significantly deviate from those encountered during training. Moreover, issues such as dilation and aliasing arise during zoom operations. These challenges stem from a fundamental issue: training sampling deficiency. In this paper, we introduce a bootstrapping framework to address this problem. Our approach synthesizes pseudo-ground truth from novel views that align with the limited training set and reintegrates these synthesized views into the training pipeline. Experimental results demonstrate that our bootstrapping technique not only reduces artifacts but also improves quantitative metrics. Furthermore, our technique is highly adaptable, allowing various Gaussian-based method to benefit from its integration.

Problem

• Addresses training sampling deficiency in 3D-GS.
• Reduces artifacts and improves rendering quality.
• Enhances adaptability for Gaussian-based methods.

Innovation

• Bootstrapping framework synthesizes pseudo-ground truth
• Reintegrates synthesized views into training pipeline
• Improves 3D Gaussian Splatting rendering quality