Stereo-LiDAR Fusion by Semi-Global Matching With Discrete Disparity-Matching Cost and Semidensification

2025-04-07

IEEE Robotics and Automation Letters

AI Summary

This paper addresses the real-time, non-learning-based fusion of LiDAR and stereo camera depth estimation in robotics and automation. To tackle challenges in accuracy, robustness, and computational efficiency, we propose a tightly coupled深度融合 method. Our approach introduces: (1) a discrete disparity cost (DDC) formulation to accelerate stereo matching; (2) a LiDAR semi-densification strategy combined with multi-source geometric consistency checks for cross-modal confidence self-calibration; and (3) GPU-accelerated parallelization of Semi-Global Matching (SGM), disparity cost discretization, and point cloud reconstruction. Evaluated on the KITTI benchmark, our method achieves an end-to-end depth error of 2.79%, outperforming the current best real-time method (3.05%). Moreover, it maintains stable performance under challenging conditions—including sparse LiDAR returns, varying weather, and indoor environments—demonstrating superior generalizability and robustness.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

We present a real-time, non-learning depth estimation method that fuses Light Detection and Ranging (LiDAR) data with stereo camera input. Our approach comprises three key techniques: Semi-Global Matching (SGM) stereo with Discrete Disparity-matching Cost (DDC), semidensification of LiDAR disparity, and a consistency check that combines stereo images and LiDAR data. Each of these components is designed for parallelization on a GPU to realize real-time performance. When it was evaluated on the KITTI dataset, the proposed method achieved an error rate of 2.79%, outperforming the previous state-of-the-art real-time stereo-LiDAR fusion method, which had an error rate of 3.05%. Furthermore, we tested the proposed method in various scenarios, including different LiDAR point densities, varying weather conditions, and indoor environments, to demonstrate its high adaptability. We believe that the real-time and non-learning nature of our method makes it highly practical for applications in robotics and automation.

Problem

• Real-time depth estimation using stereo-LiDAR fusion
• Improving accuracy over existing fusion methods
• Adapting to diverse environmental conditions robustly

Innovation

• Semi-Global Matching with Discrete Disparity Cost
• Semidensification of LiDAR disparity data
• GPU-accelerated consistency check fusion