Spiking Neural Network as Adaptive Event Stream Slicer

2024-10-03

Neural Information Processing Systems

AI Summary

Existing event-camera algorithms commonly employ fixed-time-window slicing of event streams, leading to critical temporal information loss in both high- and low-speed motion scenarios. To address this, we propose SpikeSlicer—a plug-and-play, adaptive event-stream slicing method that introduces a lightweight spiking neural network (SNN) as a dynamic trigger for real-time generation of slice boundaries based on spatiotemporal event dynamics. We establish an SNN–ANN collaborative paradigm: the SNN serves as a low-power preprocessing front-end, while the ANN handles high-level tasks. Furthermore, we design a position-aware spike loss (SPA-Loss) and a feedback-based parameter update mechanism to enable end-to-end differentiable optimization. Extensive experiments on event-driven object tracking and recognition demonstrate that SpikeSlicer significantly enhances robustness across diverse motion speeds. The implementation is publicly available.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Motion & Tracking
• Machine Learning: Time-Series/Data Streams

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Event-based cameras are attracting significant interest as they provide rich edge information, high dynamic range, and high temporal resolution. Many state-of-the-art event-based algorithms rely on splitting the events into fixed groups, resulting in the omission of crucial temporal information, particularly when dealing with diverse motion scenarios (eg, high/low speed).In this work, we propose SpikeSlicer, a novel-designed plug-and-play event processing method capable of splitting events stream adaptively.SpikeSlicer utilizes a low-energy spiking neural network (SNN) to trigger event slicing. To guide the SNN to fire spikes at optimal time steps, we propose the Spiking Position-aware Loss (SPA-Loss) to modulate the neuron's state. Additionally, we develop a Feedback-Update training strategy that refines the slicing decisions using feedback from the downstream artificial neural network (ANN). Extensive experiments demonstrate that our method yields significant performance improvements in event-based object tracking and recognition. Notably, SpikeSlicer provides a brand-new SNN-ANN cooperation paradigm, where the SNN acts as an efficient, low-energy data processor to assist the ANN in improving downstream performance, injecting new perspectives and potential avenues of exploration. Our code is available at https://github.com/AndyCao1125/SpikeSlicer.

Problem

• Adaptive event stream slicing for diverse motion scenarios
• Low-energy SNN triggering optimal event segmentation
• SNN-ANN cooperation enhancing downstream task performance

Innovation

• Adaptive event stream slicing using SNN
• Spiking Position-aware Loss for optimal timing
• Feedback-Update strategy for slicing refinement