RLCNet: An end-to-end deep learning framework for simultaneous online calibration of LiDAR, RADAR, and Camera

2025-12-09

AI Summary

To address extrinsic calibration degradation in autonomous driving caused by mechanical vibrations and dynamic sensor drift across LiDAR, RADAR, and camera modalities, this paper proposes RLCNet—a fully end-to-end trainable online calibration framework. RLCNet fuses multi-modal features and incorporates a weighted moving average filter with adaptive outlier suppression to enable real-time, dynamic optimization of extrinsic parameters. Unlike static or offline calibration approaches, RLCNet significantly enhances robustness against sensor drift, reduces prediction noise, and supports millisecond-level parameter updates. Experiments on real-world road datasets demonstrate that RLCNet achieves superior extrinsic parameter estimation accuracy compared to current state-of-the-art methods, while maintaining strong real-time performance and practical deployability in production systems.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Accurate extrinsic calibration of LiDAR, RADAR, and camera sensors is essential for reliable perception in autonomous vehicles. Still, it remains challenging due to factors such as mechanical vibrations and cumulative sensor drift in dynamic environments. This paper presents RLCNet, a novel end-to-end trainable deep learning framework for the simultaneous online calibration of these multimodal sensors. Validated on real-world datasets, RLCNet is designed for practical deployment and demonstrates robust performance under diverse conditions. To support real-time operation, an online calibration framework is introduced that incorporates a weighted moving average and outlier rejection, enabling dynamic adjustment of calibration parameters with reduced prediction noise and improved resilience to drift. An ablation study highlights the significance of architectural choices, while comparisons with existing methods demonstrate the superior accuracy and robustness of the proposed approach.

Problem

• Simultaneous online calibration of LiDAR, RADAR, and camera sensors
• Addressing sensor drift and vibrations in dynamic environments
• Enabling real-time, robust calibration for autonomous vehicles

Innovation

• End-to-end deep learning for simultaneous sensor calibration
• Online framework with weighted moving average and outlier rejection
• Real-time dynamic adjustment reducing noise and drift