Keypoint-based Dynamic Object 6-DoF Pose Tracking via Event Camera

2026-04-25

AI Summary

This work addresses the challenge of 6-degree-of-freedom pose estimation for dynamic objects under adverse conditions such as motion blur, sensor noise, and low illumination by proposing a novel event camera–based approach. Leveraging the high dynamic range and low latency of event cameras, the method extracts keypoints from time surfaces and achieves robust tracking by integrating event polarity, spatial coordinates, and local event density—a contribution introduced here for the first time in keypoint tracking. Furthermore, an efficient 2D–3D keypoint hashing scheme is designed to enable lightweight correspondence mapping, which, combined with the EPnP algorithm, yields accurate pose estimates. Evaluated on both synthetic and real-world datasets, the proposed method significantly outperforms existing approaches, achieving notable advances in both accuracy and robustness.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Tracking, profiling, and countermeasures against them
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Accurate 6-DoF pose estimation of objects is critical for robots to perform precise manipulation tasks. However, for dynamic object pose estimation, conventional camera-based approaches face several major challenges, such as motion blur, sensor noise, and low-light limitation. To address these issues, we employ event cameras, whose high dynamic range and low latency offer a promising solution. Furthermore, we propose a keypoint-based detection and tracking approach for dynamic object pose estimation. Firstly, a keypoint detection network is constructed to extract keypoints from the time surface generated by the event stream. Subsequently, the polarity and spatial coordinates of the events are leveraged, and the event density in the vicinity of each keypoint is utilized to achieve continuous keypoint tracking. Finally, a hash mapping is established between the 2D keypoints and the 3D model keypoints, and the EPnP algorithm is employed to estimate the 6-DoF pose. Experimental results demonstrate that, whether in simulated or real event environments, the proposed method outperforms the event-based state-of-the-art methods in terms of both accuracy and robustness.

Problem

• 6-DoF pose estimation
• dynamic object
• event camera
• motion blur
• low-light

Innovation

• event camera
• 6-DoF pose estimation
• keypoint tracking
• time surface
• EPnP