Event-Based Visual Teach-and-Repeat via Fast Fourier-Domain Cross-Correlation

2025-09-21

AI Summary

To address the inherent latency of conventional frame-based cameras—caused by fixed frame rates (30–60 Hz)—this paper proposes the first event-camera-based visual teach-and-repeat navigation system. Our method introduces a novel frequency-domain cross-correlation matching framework that exploits the binary nature of event streams and efficient Fourier-domain computation. Coupled with lightweight binarization and compression acceleration strategies, it achieves real-time processing at over 300 Hz without compromising localization accuracy. Evaluated on a Prophesee EVK4 HD event camera integrated with an AgileX Scout Mini platform across >4,000 meters of indoor and outdoor trajectories, the system attains an absolute position error <24 cm and control update rates exceeding 300 Hz. These results significantly outperform frame-based approaches and represent the first demonstration of high-frequency, low-latency, high-precision event-driven teach-and-repeat navigation.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Localization, Mapping, and Navigation
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Visual teach-and-repeat navigation enables robots to autonomously traverse previously demonstrated paths by comparing current sensory input with recorded trajectories. However, conventional frame-based cameras fundamentally limit system responsiveness: their fixed frame rates (typically 30-60 Hz) create inherent latency between environmental changes and control responses. Here we present the first event-camera-based visual teach-and-repeat system. To achieve this, we develop a frequency-domain cross-correlation framework that transforms the event stream matching problem into computationally efficient Fourier space multiplications, capable of exceeding 300Hz processing rates, an order of magnitude faster than frame-based approaches. By exploiting the binary nature of event frames and applying image compression techniques, we further enhance the computational speed of the cross-correlation process without sacrificing localization accuracy. Extensive experiments using a Prophesee EVK4 HD event camera mounted on an AgileX Scout Mini robot demonstrate successful autonomous navigation across 4000+ meters of indoor and outdoor trajectories. Our system achieves ATEs below 24 cm while maintaining consistent high-frequency control updates. Our evaluations show that our approach achieves substantially higher update rates compared to conventional frame-based systems, underscoring the practical viability of event-based perception for real-time robotic navigation.

Problem

• Overcoming latency limitations of frame-based cameras in robot navigation
• Enabling real-time visual matching for autonomous path following
• Achieving high-frequency processing rates exceeding 300Hz for robotic control

Innovation

• Event-camera-based visual navigation system
• Frequency-domain cross-correlation for event streams
• Binary event frame compression for efficiency