L2Calib: $SE(3)$-Manifold Reinforcement Learning for Robust Extrinsic Calibration with Degenerate Motion Resilience

2025-08-08

AI Summary

Existing extrinsic calibration methods rely on structured targets or strong ego-motion excitation, limiting their applicability in real-world scenarios; online calibration methods, in contrast, often fail under weak excitation. This paper proposes the first reinforcement learning–based framework for online extrinsic calibration, formulating calibration as a sequential decision-making problem over $SE(3)$ pose optimization. We innovatively model 3D rotations using the Bingham distribution to rigorously preserve quaternion antipodal symmetry. A trajectory-alignment reward function and an automatic data filtering module are designed to ensure robust convergence without structured targets or strong motion. The method requires no prior knowledge of initial extrinsics and supports diverse platforms—including UAVs, autonomous vehicles, and handheld devices—using only routine operational data. Experimental results demonstrate superior accuracy, convergence stability, and generalizability compared to conventional approaches.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: State Estimation
• Computer Vision: Learning & Optimization for CV

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Extrinsic calibration is essential for multi-sensor fusion, existing methods rely on structured targets or fully-excited data, limiting real-world applicability. Online calibration further suffers from weak excitation, leading to unreliable estimates. To address these limitations, we propose a reinforcement learning (RL)-based extrinsic calibration framework that formulates extrinsic calibration as a decision-making problem, directly optimizes $SE(3)$ extrinsics to enhance odometry accuracy. Our approach leverages a probabilistic Bingham distribution to model 3D rotations, ensuring stable optimization while inherently retaining quaternion symmetry. A trajectory alignment reward mechanism enables robust calibration without structured targets by quantitatively evaluating estimated tightly-coupled trajectory against a reference trajectory. Additionally, an automated data selection module filters uninformative samples, significantly improving efficiency and scalability for large-scale datasets. Extensive experiments on UAVs, UGVs, and handheld platforms demonstrate that our method outperforms traditional optimization-based approaches, achieving high-precision calibration even under weak excitation conditions. Our framework simplifies deployment on diverse robotic platforms by eliminating the need for high-quality initial extrinsics and enabling calibration from routine operating data. The code is available at https://github.com/APRIL-ZJU/learn-to-calibrate.

Problem

• Robust extrinsic calibration for multi-sensor fusion
• Overcoming weak excitation in online calibration
• Eliminating need for structured targets or full excitation

Innovation

• Reinforcement learning optimizes SE(3) extrinsics directly
• Bingham distribution models 3D rotations stably
• Automated data selection improves efficiency and scalability