Towards Low-Latency Event Stream-based Visual Object Tracking: A Slow-Fast Approach

2025-05-19

AI Summary

To address the performance bottlenecks of low-frame-rate RGB camera-based trackers in latency-critical and resource-constrained scenarios, this paper proposes SFTrack, an event-camera-driven slow-fast dual-modal tracking framework. The slow tracker ensures high accuracy, while the fast tracker employs a lightweight graph neural network backbone, FlashAttention, and a single-forward multi-box detection head to achieve millisecond-level latency. We introduce, for the first time, an event-stream graph representation learning scheme and a dual-FlashAttention collaborative architecture, enabling seamless trade-off between accuracy and latency. The fast tracker is jointly optimized via supervised fine-tuning and knowledge distillation from the slow tracker. Evaluated on FE240, COESOT, and EventVOT benchmarks, the fast tracker reduces latency by 62% while maintaining accuracy close to that of the slow tracker, significantly improving the real-time accuracy–latency trade-off. The code is publicly available.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Learning on the Edge & Model Compression

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Tracking, profiling, and countermeasures against them
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Existing tracking algorithms typically rely on low-frame-rate RGB cameras coupled with computationally intensive deep neural network architectures to achieve effective tracking. However, such frame-based methods inherently face challenges in achieving low-latency performance and often fail in resource-constrained environments. Visual object tracking using bio-inspired event cameras has emerged as a promising research direction in recent years, offering distinct advantages for low-latency applications. In this paper, we propose a novel Slow-Fast Tracking paradigm that flexibly adapts to different operational requirements, termed SFTrack. The proposed framework supports two complementary modes, i.e., a high-precision slow tracker for scenarios with sufficient computational resources, and an efficient fast tracker tailored for latency-aware, resource-constrained environments. Specifically, our framework first performs graph-based representation learning from high-temporal-resolution event streams, and then integrates the learned graph-structured information into two FlashAttention-based vision backbones, yielding the slow and fast trackers, respectively. The fast tracker achieves low latency through a lightweight network design and by producing multiple bounding box outputs in a single forward pass. Finally, we seamlessly combine both trackers via supervised fine-tuning and further enhance the fast tracker's performance through a knowledge distillation strategy. Extensive experiments on public benchmarks, including FE240, COESOT, and EventVOT, demonstrate the effectiveness and efficiency of our proposed method across different real-world scenarios. The source code has been released on https://github.com/Event-AHU/SlowFast_Event_Track.

Problem

• Achieving low-latency visual object tracking with event cameras
• Overcoming resource constraints in frame-based tracking methods
• Adapting tracking precision to computational resource availability

Innovation

• Slow-Fast Tracking paradigm for flexible adaptation
• Graph-based representation learning from event streams
• Lightweight network design for low latency