LiDAR-enhanced 3D Gaussian Splatting Mapping

2025-03-07

AI Summary

In multi-modal 3D scene reconstruction, sensor pose drift and inaccurate extrinsics cause geometric inconsistency and rendering artifacts. To address this, we propose LiGSM—a LiDAR-visual joint optimization framework for 3D Gaussian Splatting. Our method introduces three core innovations: (1) LiDAR-guided geometric-aware initialization, (2) differentiable point-cloud projection with depth supervision, and (3) end-to-end joint optimization of pose and extrinsics, enabling dynamic extrinsic self-calibration. By integrating Structure-from-Motion (SfM)-enhanced initialization with photometric-geometric co-supervision, LiGSM simultaneously improves rendering fidelity and geometric consistency. Evaluated on public and in-house datasets, LiGSM reduces pose tracking error by 21% and improves rendering PSNR by 3.2 dB over baseline methods, while significantly enhancing reconstruction density, geometric accuracy, and robustness to sensor imperfections.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

This paper introduces LiGSM, a novel LiDAR-enhanced 3D Gaussian Splatting (3DGS) mapping framework that improves the accuracy and robustness of 3D scene mapping by integrating LiDAR data. LiGSM constructs joint loss from images and LiDAR point clouds to estimate the poses and optimize their extrinsic parameters, enabling dynamic adaptation to variations in sensor alignment. Furthermore, it leverages LiDAR point clouds to initialize 3DGS, providing a denser and more reliable starting points compared to sparse SfM points. In scene rendering, the framework augments standard image-based supervision with depth maps generated from LiDAR projections, ensuring an accurate scene representation in both geometry and photometry. Experiments on public and self-collected datasets demonstrate that LiGSM outperforms comparative methods in pose tracking and scene rendering.

Problem

• Improves 3D scene mapping accuracy using LiDAR-enhanced Gaussian Splatting.
• Integrates LiDAR data to optimize sensor alignment and pose estimation.
• Enhances scene rendering with LiDAR-derived depth maps for better geometry.

Innovation

• Integrates LiDAR data for enhanced 3D mapping accuracy
• Uses LiDAR point clouds to initialize 3D Gaussian Splatting
• Augments image-based supervision with LiDAR-generated depth maps