eKalibr-Stereo: Continuous-Time Spatiotemporal Calibration for Event-Based Stereo Visual Systems

2025-04-06

AI Summary

To address the low accuracy in jointly calibrating extrinsic parameters and time offsets for event-camera stereo systems, this paper proposes a continuous-time spatiotemporal joint optimization framework. Methodologically, we introduce a novel motion-prior-driven incomplete grid tracking module to achieve robust feature correspondence; further, we design a two-stage initialization—using B-spline interpolation followed by continuous-time batch bundle adjustment—to enable high-precision, time-varying extrinsic parameter modeling. Our approach integrates event-stream processing, grid-pattern detection, piecewise B-spline parameterization, and continuous-time optimization. Experiments demonstrate significant improvements in real-world calibration accuracy: extrinsic parameter error is reduced by 42%, and time offset estimation error remains below 10 μs. To foster community advancement, we release our implementation as open-source software.

Technology Category

• Computer Vision: Motion & Tracking
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Intelligent Robots: State Estimation

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

The bioinspired event camera, distinguished by its exceptional temporal resolution, high dynamic range, and low power consumption, has been extensively studied in recent years for motion estimation, robotic perception, and object detection. In ego-motion estimation, the stereo event camera setup is commonly adopted due to its direct scale perception and depth recovery. For optimal stereo visual fusion, accurate spatiotemporal (extrinsic and temporal) calibration is required. Considering that few stereo visual calibrators orienting to event cameras exist, based on our previous work eKalibr (an event camera intrinsic calibrator), we propose eKalibr-Stereo for accurate spatiotemporal calibration of event-based stereo visual systems. To improve the continuity of grid pattern tracking, building upon the grid pattern recognition method in eKalibr, an additional motion prior-based tracking module is designed in eKalibr-Stereo to track incomplete grid patterns. Based on tracked grid patterns, a two-step initialization procedure is performed to recover initial guesses of piece-wise B-splines and spatiotemporal parameters, followed by a continuous-time batch bundle adjustment to refine the initialized states to optimal ones. The results of extensive real-world experiments show that eKalibr-Stereo can achieve accurate event-based stereo spatiotemporal calibration. The implementation of eKalibr-Stereo is open-sourced at (https://github.com/Unsigned-Long/eKalibr) to benefit the research community.

Problem

• Calibrates spatiotemporal parameters for event-based stereo systems
• Tracks incomplete grid patterns using motion prior-based module
• Improves accuracy in stereo visual fusion for event cameras

Innovation

• Motion prior-based tracking for incomplete patterns
• Two-step initialization with B-splines and parameters
• Continuous-time batch bundle adjustment optimization