RMGS-SLAM: Real-time Multi-sensor Gaussian Splatting SLAM

2026-04-14

AI Summary

This work addresses the challenge of achieving real-time, low-latency 3D Gaussian Splatting-based SLAM in large-scale real-world environments by proposing a tightly coupled LiDAR–inertial–visual (LIV) fusion framework. The system concurrently performs state estimation, Gaussian primitive initialization, and global optimization, and introduces a cascaded strategy that integrates feedforward predictions with voxel-based PCA geometric priors to enhance initialization quality. Notably, it is the first to directly employ Generalized Iterative Closest Point (GICP) on the optimized global Gaussian map for loop closure detection and pose graph optimization, substantially improving global consistency in large-scale scenes. Experiments demonstrate that the proposed system achieves an excellent balance among localization accuracy, rendering fidelity, and real-time performance on both public and self-collected large-scale outdoor datasets featuring loop closures.

Technology Category

• Intelligent Robots: State Estimation
• Computer Vision: 3D Computer Vision
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Real-time 3D Gaussian splatting (3DGS)-based Simultaneous Localization and Mapping (SLAM) in large-scale real-world environments remains challenging, as existing methods often struggle to jointly achieve low-latency pose estimation, 3D Gaussian reconstruction in step with incoming sensor streams, and long-term global consistency. In this paper, we present a tightly coupled LiDAR-Inertial-Visual (LIV) 3DGS-based SLAM framework for real-time pose estimation and photorealistic mapping in large-scale real-world scenes. The system executes state estimation and 3D Gaussian primitive initialization in parallel with global Gaussian optimization, thereby enabling continuous dense mapping. To improve Gaussian initialization quality and accelerate optimization convergence, we introduce a cascaded strategy that combines feed-forward predictions with voxel-based principal component analysis (voxel-PCA) geometric priors. To enhance global consistency in large scenes, we further perform loop closure directly on the optimized global Gaussian map by estimating loop constraints through Gaussian-based Generalized Iterative Closest Point (GICP) registration, followed by pose-graph optimization. In addition, we collected challenging large-scale looped outdoor SLAM sequences with hardware-synchronized LiDAR-camera-IMU and ground-truth trajectories to support realistic and comprehensive evaluation. Extensive experiments on both public datasets and our dataset demonstrate that the proposed method achieves a strong balance among real-time efficiency, localization accuracy, and rendering quality across diverse and challenging real-world scenes.

Problem

• 3D Gaussian Splatting
• SLAM
• real-time
• large-scale
• global consistency

Innovation

• 3D Gaussian Splatting
• LiDAR-Inertial-Visual SLAM
• real-time dense mapping
• loop closure
• voxel-PCA