ILIF: Temporal Inhibitory Leaky Integrate-and-Fire Neuron for Overactivation in Spiking Neural Networks

2025-05-15

AI Summary

To address the “gamma dilemma” in spiking neural networks (SNNs)—where large γ in surrogate gradient methods causes neuronal over-activation and high energy consumption, while small γ leads to vanishing gradients and weakened temporal dynamics—this paper proposes a temporally suppressed Leaky Integrate-and-Fire (LIF) neuron model. It introduces, for the first time, a biologically inspired inhibitory mechanism coupling membrane potential and synaptic current, thereby decoupling activation strength control from gradient propagation requirements at the mechanistic level. Theoretically, the model guarantees both training stability and energy efficiency. Built upon the LIF framework, it extends inhibitory units compatible with end-to-end backpropagation and provides convergence analysis. Experiments demonstrate an average 37% reduction in spike rate, 1.2–2.8% improvement in classification accuracy, and enhanced training robustness across multiple benchmark datasets. The implementation is open-sourced.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Bio-inspired Learning
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The Spiking Neural Network (SNN) has drawn increasing attention for its energy-efficient, event-driven processing and biological plausibility. To train SNNs via backpropagation, surrogate gradients are used to approximate the non-differentiable spike function, but they only maintain nonzero derivatives within a narrow range of membrane potentials near the firing threshold, referred to as the surrogate gradient support width gamma. We identify a major challenge, termed the dilemma of gamma: a relatively large gamma leads to overactivation, characterized by excessive neuron firing, which in turn increases energy consumption, whereas a small gamma causes vanishing gradients and weakens temporal dependencies. To address this, we propose a temporal Inhibitory Leaky Integrate-and-Fire (ILIF) neuron model, inspired by biological inhibitory mechanisms. This model incorporates interconnected inhibitory units for membrane potential and current, effectively mitigating overactivation while preserving gradient propagation. Theoretical analysis demonstrates ILIF effectiveness in overcoming the gamma dilemma, and extensive experiments on multiple datasets show that ILIF improves energy efficiency by reducing firing rates, stabilizes training, and enhances accuracy. The code is available at github.com/kaisun1/ILIF.

Problem

• Balancing surrogate gradient width to prevent overactivation and vanishing gradients
• Reducing excessive neuron firing to improve energy efficiency in SNNs
• Enhancing temporal dependencies while maintaining stable training in SNNs

Innovation

• Proposes temporal Inhibitory Leaky Integrate-and-Fire (ILIF) neuron
• Uses interconnected inhibitory units for membrane potential
• Mitigates overactivation while preserving gradient propagation