Hydra: Marker-Free RGB-D Hand-Eye Calibration

2025-04-29

AI Summary

This paper addresses the challenging problem of markerless RGB-D hand-eye calibration. We propose a robust point-to-plane ICP optimization method formulated directly in Lie algebra space. Our approach requires only three arbitrarily sampled robot end-effector poses—eliminating reliance on artificial calibration markers. The key contribution is the first formulation of a geometrically consistent and differentiable point-to-plane distance objective function on the Lie algebra, enabling stable gradient-based optimization. Experiments demonstrate a 90% calibration success rate, an average runtime of 0.8 ± 0.4 s (two orders of magnitude faster than baseline methods), and task-space accuracy of 5 mm (surpassing the 7 mm achieved by classical approaches); global convergence rate improves by 2–3×. To foster reproducibility and deployment, we release an open-source dataset, implementation, and a ROS 2 interface supporting plug-and-play integration.

Technology Category

• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Biometrics, Face, Gesture & Pose

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

This work presents an RGB-D imaging-based approach to marker-free hand-eye calibration using a novel implementation of the iterative closest point (ICP) algorithm with a robust point-to-plane (PTP) objective formulated on a Lie algebra. Its applicability is demonstrated through comprehensive experiments using three well known serial manipulators and two RGB-D cameras. With only three randomly chosen robot configurations, our approach achieves approximately 90% successful calibrations, demonstrating 2-3x higher convergence rates to the global optimum compared to both marker-based and marker-free baselines. We also report 2 orders of magnitude faster convergence time (0.8 +/- 0.4 s) for 9 robot configurations over other marker-free methods. Our method exhibits significantly improved accuracy (5 mm in task space) over classical approaches (7 mm in task space) whilst being marker-free. The benchmarking dataset and code are open sourced under Apache 2.0 License, and a ROS 2 integration with robot abstraction is provided to facilitate deployment.

Problem

• Marker-free hand-eye calibration using RGB-D and ICP
• Improved accuracy and speed over classical methods
• Open-source dataset and ROS 2 integration provided

Innovation

• Marker-free RGB-D hand-eye calibration using ICP
• Lie algebra-based robust point-to-plane objective
• Fast convergence and high accuracy without markers