GaussianRoom: Improving 3D Gaussian Splatting with SDF Guidance and Monocular Cues for Indoor Scene Reconstruction

2024-05-30

arXiv.org

AI Summary

To address geometric incompleteness and significant noise in large-scale textureless indoor scenes—caused by poor point cloud initialization and insufficient optimization in 3D Gaussian Splatting (3DGS)—this paper proposes a neural Signed Distance Function (SDF)-guided co-optimization framework. Our method jointly optimizes implicit SDF representations and 3DGS parameters for the first time, introduces normal- and edge-driven geometric regularization to resolve reconstruction ambiguities in textureless regions, and designs density-adaptive control and sampling-guided strategies to enhance surface consistency. Evaluated on ScanNet and ScanNet++, our approach achieves state-of-the-art performance in both surface reconstruction accuracy and novel-view synthesis quality. It significantly improves geometric completeness and detail fidelity for large-scale textureless indoor scenes while preserving real-time rendering capability.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Learning with Manifolds

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Embodied intelligence requires precise reconstruction and rendering to simulate large-scale real-world data. Although 3D Gaussian Splatting (3DGS) has recently demonstrated high-quality results with real-time performance, it still faces challenges in indoor scenes with large, textureless regions, resulting in incomplete and noisy reconstructions due to poor point cloud initialization and underconstrained optimization. Inspired by the continuity of signed distance field (SDF), which naturally has advantages in modeling surfaces, we propose a unified optimization framework that integrates neural signed distance fields (SDFs) with 3DGS for accurate geometry reconstruction and real-time rendering. This framework incorporates a neural SDF field to guide the densification and pruning of Gaussians, enabling Gaussians to model scenes accurately even with poor initialized point clouds. Simultaneously, the geometry represented by Gaussians improves the efficiency of the SDF field by piloting its point sampling. Additionally, we introduce two regularization terms based on normal and edge priors to resolve geometric ambiguities in textureless areas and enhance detail accuracy. Extensive experiments in ScanNet and ScanNet++ show that our method achieves state-of-the-art performance in both surface reconstruction and novel view synthesis.

Problem

• Improving 3D Gaussian Splatting for indoor scenes with SDF guidance
• Addressing incomplete noisy reconstructions in textureless regions
• Enhancing detail accuracy using normal and edge priors

Innovation

• Integrates neural SDFs with 3DGS for accurate geometry
• Uses SDF-guided densification and pruning of Gaussians
• Adds normal and edge priors for textureless areas