GS-LIVO: Real-Time LiDAR, Inertial, and Visual Multi-sensor Fused Odometry with Gaussian Mapping

2025-01-15

AI Summary

Existing 3D Gaussian splatting SLAM methods suffer from insufficient point cloud density, poor occlusion robustness, and limited real-time performance on embedded platforms. To address these limitations, this paper proposes the first real-time Gaussian fusion SLAM system specifically designed for embedded devices. Methodologically, we introduce a novel hybrid mapping architecture combining a sliding-window Gaussian map with a recursive octree-hashed voxel-based global map; further, we design an iterated extended square-root Kalman filter (IESKF)-based tightly coupled multi-sensor odometry that fuses LiDAR, IMU, and visual data, augmented by photometric gradient optimization and multi-source spatiotemporal synchronization. Our contributions include significantly reduced GPU memory footprint and computational overhead, enabling dense mapping and localization at over 30 FPS on the Jetson Orin NX; additionally, the system achieves adaptive high-fidelity reconstruction of dynamic scene details and strong robustness against severe occlusions.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: 3D Computer Vision
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

In recent years, 3D Gaussian splatting (3D-GS) has emerged as a novel scene representation approach. However, existing vision-only 3D-GS methods often rely on hand-crafted heuristics for point-cloud densification and face challenges in handling occlusions and high GPU memory and computation consumption. LiDAR-Inertial-Visual (LIV) sensor configuration has demonstrated superior performance in localization and dense mapping by leveraging complementary sensing characteristics: rich texture information from cameras, precise geometric measurements from LiDAR, and high-frequency motion data from IMU. Inspired by this, we propose a novel real-time Gaussian-based simultaneous localization and mapping (SLAM) system. Our map system comprises a global Gaussian map and a sliding window of Gaussians, along with an IESKF-based odometry. The global Gaussian map consists of hash-indexed voxels organized in a recursive octree, effectively covering sparse spatial volumes while adapting to different levels of detail and scales. The Gaussian map is initialized through multi-sensor fusion and optimized with photometric gradients. Our system incrementally maintains a sliding window of Gaussians, significantly reducing GPU computation and memory consumption by only optimizing the map within the sliding window. Moreover, we implement a tightly coupled multi-sensor fusion odometry with an iterative error state Kalman filter (IESKF), leveraging real-time updating and rendering of the Gaussian map. Our system represents the first real-time Gaussian-based SLAM framework deployable on resource-constrained embedded systems, demonstrated on the NVIDIA Jetson Orin NX platform. The framework achieves real-time performance while maintaining robust multi-sensor fusion capabilities. All implementation algorithms, hardware designs, and CAD models will be publicly available.

Problem

• 3D Gaussian Splatting
• Visual Sensor Limitations
• Resource Consumption

Innovation

• Real-time SLAM
• Multi-sensor Fusion
• GPU Optimization