SplatMAP: Online Dense Monocular SLAM with 3D Gaussian Splatting

2025-01-13

AI Summary

To address the trade-off among geometric accuracy, detail fidelity, and computational efficiency in monocular real-time 3D reconstruction, this paper proposes a SLAM-guided adaptive 3D Gaussian Splatting (3DGS) framework. Methodologically, it pioneers the integration of dense monocular SLAM’s depth and pose estimates into the 3DGS optimization pipeline, enabling joint photometric-geometric optimization driven by geometric constraints. It further introduces edge-aware regularization and dynamic point-cloud-guided Gaussian parameter updates. An adaptive Gaussian densification strategy enhances structural completeness and texture detail. Evaluated on Replica and TUM-RGBD datasets, the method achieves PSNR = 36.864 (+10.7%), SSIM = 0.985 (+6.4%), and LPIPS = 0.040 (−49.4%)—outperforming all existing monocular approaches. The framework enables online dense reconstruction with high fidelity, real-time capability, and strong geometric consistency.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Learning with Manifolds

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Achieving high-fidelity 3D reconstruction from monocular video remains challenging due to the inherent limitations of traditional methods like Structure-from-Motion (SfM) and monocular SLAM in accurately capturing scene details. While differentiable rendering techniques such as Neural Radiance Fields (NeRF) address some of these challenges, their high computational costs make them unsuitable for real-time applications. Additionally, existing 3D Gaussian Splatting (3DGS) methods often focus on photometric consistency, neglecting geometric accuracy and failing to exploit SLAM's dynamic depth and pose updates for scene refinement. We propose a framework integrating dense SLAM with 3DGS for real-time, high-fidelity dense reconstruction. Our approach introduces SLAM-Informed Adaptive Densification, which dynamically updates and densifies the Gaussian model by leveraging dense point clouds from SLAM. Additionally, we incorporate Geometry-Guided Optimization, which combines edge-aware geometric constraints and photometric consistency to jointly optimize the appearance and geometry of the 3DGS scene representation, enabling detailed and accurate SLAM mapping reconstruction. Experiments on the Replica and TUM-RGBD datasets demonstrate the effectiveness of our approach, achieving state-of-the-art results among monocular systems. Specifically, our method achieves a PSNR of 36.864, SSIM of 0.985, and LPIPS of 0.040 on Replica, representing improvements of 10.7%, 6.4%, and 49.4%, respectively, over the previous SOTA. On TUM-RGBD, our method outperforms the closest baseline by 10.2%, 6.6%, and 34.7% in the same metrics. These results highlight the potential of our framework in bridging the gap between photometric and geometric dense 3D scene representations, paving the way for practical and efficient monocular dense reconstruction.

Problem

• 3D Scene Reconstruction
• Shape Accuracy
• Real-time Application

Innovation

• Dense Monocular SLAM
• 3D Geometric Optimization
• Real-time Scene Reconstruction