Optical Flow-Guided 6DoF Object Pose Tracking with an Event Camera

2025-12-24

AI Summary

Traditional cameras suffer from motion blur, illumination changes, occlusions, and noise in 6DoF object pose tracking under high-dynamic scenes. To address this, we propose an event-camera-based optical flow-guided 2D–3D hybrid feature matching and iterative optimization method. Our key contributions are: (1) the first event-driven corner detection mechanism that maximizes optical flow probability; (2) a novel corner–edge association paradigm explicitly constrained by optical flow; and (3) joint spatiotemporal modeling of event streams with nonlinear pose optimization. Evaluated on both synthetic and real-world event datasets, our method significantly outperforms existing state-of-the-art approaches, achieving superior pose accuracy and enhanced robustness to rapid motion, illumination variations, and partial occlusions.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Search and Optimization: Learning to Search

Application Category

• Security and Privacy: Tracking, profiling, and countermeasures against them
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Object pose tracking is one of the pivotal technologies in multimedia, attracting ever-growing attention in recent years. Existing methods employing traditional cameras encounter numerous challenges such as motion blur, sensor noise, partial occlusion, and changing lighting conditions. The emerging bio-inspired sensors, particularly event cameras, possess advantages such as high dynamic range and low latency, which hold the potential to address the aforementioned challenges. In this work, we present an optical flow-guided 6DoF object pose tracking method with an event camera. A 2D-3D hybrid feature extraction strategy is firstly utilized to detect corners and edges from events and object models, which characterizes object motion precisely. Then, we search for the optical flow of corners by maximizing the event-associated probability within a spatio-temporal window, and establish the correlation between corners and edges guided by optical flow. Furthermore, by minimizing the distances between corners and edges, the 6DoF object pose is iteratively optimized to achieve continuous pose tracking. Experimental results of both simulated and real events demonstrate that our methods outperform event-based state-of-the-art methods in terms of both accuracy and robustness.

Problem

• Tracking 6DoF object pose with event cameras
• Overcoming motion blur and lighting challenges
• Optimizing pose via optical flow and hybrid features

Innovation

• Event camera with high dynamic range
• Optical flow-guided 2D-3D hybrid feature extraction
• Iterative optimization minimizing corner-edge distances