AugSplat: Radiance Field-Informed Gaussian Splatting for Sparse-View Settings

2026-06-30

AI Summary

This work addresses the limitations of 3D reconstruction under sparse-view settings, where Gaussian splatting suffers from poor initial geometry while neural radiance fields (NeRFs) offer robust reconstruction at the cost of slow inference. To bridge this gap, the authors propose leveraging synthetic views rendered from a NeRF as enhanced supervision signals, integrating them into the Gaussian splatting optimization pipeline through two strategies: Staged and Dual. The Staged approach yields optimal performance, whereas the Dual strategy maintains real-image supervision throughout training. Both methods preserve real-time rendering capabilities while significantly improving reconstruction quality over standard Gaussian splatting on sparse scenes from the mip-NeRF 360 benchmark, effectively combining the strengths of NeRF’s accuracy and splatting’s efficiency.

Abstract

Generating high-quality novel views at real-time frame rates remains a central challenge in 3D vision, particularly in sparse-view scenarios. Neural radiance fields have demonstrated robust reconstruction from limited observations, but their reliance on volumetric rendering leads to high computational cost and slow inference. In contrast, Gaussian Splatting methods achieve real-time rendering through rasterization, but their optimization is highly sensitive to the quality of the initial geometry. This sensitivity becomes especially problematic in sparse-view settings, where limited observations often lead to incomplete or noisy point-cloud reconstructions. In this work, we present AugSplat, a simple framework for improving Gaussian Splatting in sparse-view regimes using radiance-field-based view augmentation. We first train a radiance field on the sparse input views and use it to synthesize additional images from nearby novel viewpoints, increasing the effective view-space coverage available for supervision. These synthetic views are then used as auxiliary supervision during Gaussian Splatting optimization. We study two variants: Staged AugSplat, which uses synthetic views for an initial optimization phase before switching to real images, and Dual AugSplat, which jointly trains on real and synthetic views with a decaying synthetic loss weight. Experiments on sparse-view mip-NeRF 360 scenes show that AugSplat improves reconstruction quality over standard Gaussian Splatting. Staged AugSplat achieves the strongest average performance, while Dual AugSplat provides a closely performing formulation that keeps real-image supervision active throughout training, and both variants preserve real-time rendering at inference.

Problem

• Gaussian Splatting
• sparse-view
• novel view synthesis
• 3D reconstruction
• radiance fields

Innovation

• Gaussian Splatting
• Neural Radiance Fields
• Sparse-View Reconstruction
• View Synthesis
• Real-Time Rendering