Sparse Convolutional Recurrent Learning for Efficient Event-based Neuromorphic Object Detection

2025-06-16

AI Summary

Event cameras generate sparse, asynchronous data, posing significant computational and energy-efficiency challenges for long-sequence spatiotemporal reasoning in edge-based object detection. Method: This paper proposes an efficient neuromorphic detection framework featuring a novel sparse convolutional recurrent learning mechanism (achieving >92% recurrent activation sparsity) and a hardware-aware SEED architecture that jointly integrates sparse tensor computation, event-driven RNNs, and hardware-software co-optimization. Contribution/Results: It is the first work to unify high spatiotemporal representation capability with low latency and high energy efficiency under extreme sparsity. The framework achieves state-of-the-art mAP on both Prophesee 1Mpx and Gen1 datasets while drastically reducing synaptic operations. Hardware emulation demonstrates substantial reductions in inference latency and energy consumption, validating its practicality and advancement for neuromorphic edge computing.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Multi-modal Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Leveraging the high temporal resolution and dynamic range, object detection with event cameras can enhance the performance and safety of automotive and robotics applications in real-world scenarios. However, processing sparse event data requires compute-intensive convolutional recurrent units, complicating their integration into resource-constrained edge applications. Here, we propose the Sparse Event-based Efficient Detector (SEED) for efficient event-based object detection on neuromorphic processors. We introduce sparse convolutional recurrent learning, which achieves over 92% activation sparsity in recurrent processing, vastly reducing the cost for spatiotemporal reasoning on sparse event data. We validated our method on Prophesee's 1 Mpx and Gen1 event-based object detection datasets. Notably, SEED sets a new benchmark in computational efficiency for event-based object detection which requires long-term temporal learning. Compared to state-of-the-art methods, SEED significantly reduces synaptic operations while delivering higher or same-level mAP. Our hardware simulations showcase the critical role of SEED's hardware-aware design in achieving energy-efficient and low-latency neuromorphic processing.

Problem

• Efficient object detection with sparse event data
• Reducing computational cost for spatiotemporal reasoning
• Enabling energy-efficient neuromorphic processing on edge devices

Innovation

• Sparse convolutional recurrent learning reduces costs
• Achieves 92% activation sparsity in processing
• Hardware-aware design enables energy-efficient neuromorphic processing