TSkips: Efficiency Through Explicit Temporal Delay Connections in Spiking Neural Networks

2024-11-22

arXiv.org

AI Summary

To address performance bottlenecks in Spiking Neural Networks (SNNs) applied to event-camera and speech tasks—specifically pulse attenuation and insufficient modeling of long-range spatiotemporal dependencies—this paper proposes explicit Time-Delayed Skip Connections (TSkips). TSkips embed learnable time delays into forward and backward pathways to enhance spike propagation efficiency and temporal representation capability. Furthermore, a training-free Neural Architecture Search (NAS) is integrated to automatically optimize delay configurations, enabling structure-temporal co-design without additional training overhead. Experiments demonstrate significant improvements: an 18% reduction in average end-point error on the DSEC-flow optical flow task; an 8% accuracy gain on DVS128 gesture recognition; and 8% and 16% accuracy improvements on the SHD and SSC speech recognition benchmarks, respectively. These results underscore TSkips’ effectiveness in modeling fine-grained spike timing and complex spatiotemporal patterns in SNNs.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Spiking Neural Networks (SNNs) with their bio-inspired Leaky Integrate-and-Fire (LIF) neurons inherently capture temporal information. This makes them well-suited for sequential tasks like processing event-based data from Dynamic Vision Sensors (DVS) and event-based speech tasks. Harnessing the temporal capabilities of SNNs requires mitigating vanishing spikes during training, capturing spatio-temporal patterns and enhancing precise spike timing. To address these challenges, we propose TSkips, augmenting SNN architectures with forward and backward skip connections that incorporate explicit temporal delays. These connections capture long-term spatio-temporal dependencies and facilitate better spike flow over long sequences. The introduction of TSkips creates a vast search space of possible configurations, encompassing skip positions and time delay values. To efficiently navigate this search space, this work leverages training-free Neural Architecture Search (NAS) to identify optimal network structures and corresponding delays. We demonstrate the effectiveness of our approach on four event-based datasets: DSEC-flow for optical flow estimation, DVS128 Gesture for hand gesture recognition and Spiking Heidelberg Digits (SHD) and Spiking Speech Commands (SSC) for speech recognition. Our method achieves significant improvements across these datasets: up to 18% reduction in Average Endpoint Error (AEE) on DSEC-flow, 8% increase in classification accuracy on DVS128 Gesture, and up to 8% and 16% higher classification accuracy on SHD and SSC, respectively.

Problem

• Mitigate vanishing spikes in Spiking Neural Networks training
• Capture long-term spatio-temporal dependencies in event-based data
• Optimize network structures with temporal delays efficiently

Innovation

• TSkips with explicit temporal delay connections
• Training-free Neural Architecture Search (NAS)
• Enhanced spike flow in long sequences