ELMAR: Enhancing LiDAR Detection with 4D Radar Motion Awareness and Cross-modal Uncertainty

2025-06-22

AI Summary

To address cross-modal spatiotemporal misalignment and insufficient motion information exploitation in LiDAR–4D radar fusion perception, this paper proposes a dynamic motion-aware and uncertainty-driven fusion framework. Methodologically, it introduces: (1) a dynamic motion-aware encoding module that explicitly models 4D radar’s radial velocity and motion states; (2) an instance-level cross-modal uncertainty estimation mechanism jointly quantifying alignment confidence between LiDAR and radar features; and (3) an uncertainty-guided prediction calibration strategy for robust feature fusion and detection optimization. The framework is trained end-to-end on the VoD dataset, achieving 74.89% overall mAP and 88.70% lane-specific mAP—both state-of-the-art—and an inference speed of 30.02 FPS. This work marks the first unified integration of 4D radar motion awareness and cross-modal confidence modeling in a single architecture.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Motion & Tracking
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

LiDAR and 4D radar are widely used in autonomous driving and robotics. While LiDAR provides rich spatial information, 4D radar offers velocity measurement and remains robust under adverse conditions. As a result, increasing studies have focused on the 4D radar-LiDAR fusion method to enhance the perception. However, the misalignment between different modalities is often overlooked. To address this challenge and leverage the strengths of both modalities, we propose a LiDAR detection framework enhanced by 4D radar motion status and cross-modal uncertainty. The object movement information from 4D radar is first captured using a Dynamic Motion-Aware Encoding module during feature extraction to enhance 4D radar predictions. Subsequently, the instance-wise uncertainties of bounding boxes are estimated to mitigate the cross-modal misalignment and refine the final LiDAR predictions. Extensive experiments on the View-of-Delft (VoD) dataset highlight the effectiveness of our method, achieving state-of-the-art performance with the mAP of 74.89% in the entire area and 88.70% within the driving corridor while maintaining a real-time inference speed of 30.02 FPS.

Problem

• Addressing misalignment between LiDAR and 4D radar modalities
• Enhancing LiDAR detection with 4D radar motion awareness
• Mitigating cross-modal uncertainty for improved perception

Innovation

• Dynamic Motion-Aware Encoding module
• Cross-modal uncertainty estimation
• 4D radar-LiDAR fusion enhancement