Efficient Event-based Delay Learning in Spiking Neural Networks

2025-01-13

AI Summary

To address the limitations of recurrent spiking neural networks (SNNs) in temporal modeling—including non-trainable synaptic delays, inaccurate gradient computation, and poor generalization—this paper introduces the first event-driven, learnable delay method supporting recurrent connectivity, enabling joint optimization of synaptic weights and transmission delays. Building upon the EventProp framework, we derive exact analytical gradients for delay parameters in recurrent SNNs, accommodating multi-spike dynamics and temporally precise neuronal responses. Our approach significantly enhances robustness to delay initialization and cross-task generalization. Experimentally, it achieves state-of-the-art classification accuracy on the Yin-Yang, Spiking Heidelberg Digits (SHD), and Spiking Speech Commands (SSC) benchmarks. Moreover, it reduces memory consumption by 50% compared to the best existing baseline and accelerates training by up to 26×.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

Spiking Neural Networks (SNNs) are attracting increased attention as a more energy-efficient alternative to traditional Artificial Neural Networks. Spiking neurons are stateful and intrinsically recurrent, making them well-suited for spatio-temporal tasks. However, this intrinsic memory is limited by synaptic and membrane time constants. A powerful additional mechanism are delays. In this paper, we propose a novel event-based training method for SNNs with delays, grounded in the EventProp formalism and enabling the calculation of exact gradients with respect to weights and delays. Our method supports multiple spikes per neuron and, to our best knowledge, is the first delay learning method applicable to recurrent connections. We evaluate our method on a simple sequence detection task, and the Yin-Yang, Spiking Heidelberg Digits and Spiking Speech Commands datasets, demonstrating that our algorithm can optimize delays from suboptimal initial conditions and enhance classification accuracy compared to architectures without delays. Finally, we show that our approach uses less than half the memory of the current state-of-the-art delay-learning method and is up to 26x faster.

Problem

• Spiking Neural Networks
• Weight Optimization
• Temporal Sequence Processing

Innovation

• Spiking Neural Networks
• Delay Learning
• Memory Efficiency