NGM-SLAM: Gaussian Splatting SLAM with Radiance Field Submap

2024-05-09

arXiv.org

AI Summary

To address the limitations of 3D Gaussian Splatting (3DGS)-based SLAM in large-scale scenes—particularly its constrained representational capacity and poor loop closure detection—this paper proposes the first real-time SLAM framework integrating Neural Radiance Fields (NeRF) submaps with 3DGS. Methodologically, submap-level NeRFs serve as geometric and appearance priors to guide the progressive construction, online registration, and loop closure optimization of 3DGS. This marks the first incorporation of NeRF submaps into 3DGS-SLAM, overcoming the expressiveness bottleneck inherent in per-frame Gaussian fusion. The system supports monocular, stereo, and RGB-D front-end tracking, enabling efficient large-scale mapping and robust loop closure. Experiments on real-world large-scale scenes demonstrate significant improvements in hole-filling quality and reconstruction accuracy, achieving state-of-the-art performance in both tracking and mapping while maintaining real-time rendering efficiency.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Motion & Tracking
• Machine Learning: Neuro-Symbolic Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

SLAM systems based on Gaussian Splatting have garnered attention due to their capabilities for rapid real-time rendering and high-fidelity mapping. However, current Gaussian Splatting SLAM systems usually struggle with large scene representation and lack effective loop closure detection. To address these issues, we introduce NGM-SLAM, the first 3DGS based SLAM system that utilizes neural radiance field submaps for progressive scene expression, effectively integrating the strengths of neural radiance fields and 3D Gaussian Splatting. We utilize neural radiance field submaps as supervision and achieve high-quality scene expression and online loop closure adjustments through Gaussian rendering of fused submaps. Our results on multiple real-world scenes and large-scale scene datasets demonstrate that our method can achieve accurate hole filling and high-quality scene expression, supporting monocular, stereo, and RGB-D inputs, and achieving state-of-the-art scene reconstruction and tracking performance.

Problem

• Addresses large scene representation challenges in Gaussian Splatting SLAM.
• Improves loop closure detection using neural radiance field submaps.
• Enhances scene reconstruction and tracking with multi-sensor input support.

Innovation

• Neural radiance field submaps for scene expression
• Gaussian rendering for online loop closure
• Supports monocular, stereo, and RGB-D inputs