NexusGS: Sparse View Synthesis with Epipolar Depth Priors in 3D Gaussian Splatting

2025-03-24

AI Summary

To address depth estimation inaccuracies and unstable point cloud initialization—leading to oversmoothing and overfitting in novel view synthesis under sparse-view settings—this paper proposes NexusGS, a depth-guided reconstruction method built upon 3D Gaussian Splatting. Its core innovation is the epipolar depth hub mechanism, which embeds unsupervised depth priors via epipolar geometry modeling and robust optical flow fusion, while jointly performing filter-based pruning and camera pose regularization for end-to-end differentiable point cloud initialization—eliminating reliance on hand-crafted regularization. Evaluated on multiple sparse-view benchmarks, NexusGS achieves state-of-the-art performance: depth error reduced by 32% and PSNR improved by 2.8 dB over prior methods. Moreover, the optimized point clouds demonstrate strong transferability, enhancing the performance of other downstream methods.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Non-convex Optimization
• Intelligent Robots: State Estimation

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Security and Privacy: Data transparency and provenance

Abstract

Neural Radiance Field (NeRF) and 3D Gaussian Splatting (3DGS) have noticeably advanced photo-realistic novel view synthesis using images from densely spaced camera viewpoints. However, these methods struggle in few-shot scenarios due to limited supervision. In this paper, we present NexusGS, a 3DGS-based approach that enhances novel view synthesis from sparse-view images by directly embedding depth information into point clouds, without relying on complex manual regularizations. Exploiting the inherent epipolar geometry of 3DGS, our method introduces a novel point cloud densification strategy that initializes 3DGS with a dense point cloud, reducing randomness in point placement while preventing over-smoothing and overfitting. Specifically, NexusGS comprises three key steps: Epipolar Depth Nexus, Flow-Resilient Depth Blending, and Flow-Filtered Depth Pruning. These steps leverage optical flow and camera poses to compute accurate depth maps, while mitigating the inaccuracies often associated with optical flow. By incorporating epipolar depth priors, NexusGS ensures reliable dense point cloud coverage and supports stable 3DGS training under sparse-view conditions. Experiments demonstrate that NexusGS significantly enhances depth accuracy and rendering quality, surpassing state-of-the-art methods by a considerable margin. Furthermore, we validate the superiority of our generated point clouds by substantially boosting the performance of competing methods. Project page: https://usmizuki.github.io/NexusGS/.

Problem

• Enhances sparse-view synthesis using depth priors
• Reduces randomness in 3DGS point placement
• Improves depth accuracy and rendering quality

Innovation

• Embeds depth into point clouds directly
• Uses epipolar geometry for densification
• Leverages optical flow for depth accuracy