🤖 AI Summary
Existing spiking neural network (SNN) backbones—such as Spikformer and SEW-ResNet—rely on floating-point multiplications in residual connections, resulting in high energy consumption and poor deployability on neuromorphic hardware. To address this, we propose Spikingformer, the first fully spike-driven Transformer-style backbone for SNNs. Its core innovations include: (i) a biologically plausible multi-spike (MS) residual connection that eliminates floating-point operations, and (ii) the first complete migration of self-attention into the spike domain, thereby removing all non-spiking computations. Spikingformer enables unified cross-modal modeling of images, neuromorphic event streams, and text. Evaluated on 13 heterogeneous benchmarks, it consistently outperforms state-of-the-art baselines, achieves significant power reduction, and maintains full compatibility with mainstream neuromorphic chips. This work establishes a new paradigm and benchmark for general-purpose SNN foundation models.
📝 Abstract
Spiking neural networks (SNNs) offer a promising energy-efficient alternative to artificial neural networks, due to their event-driven spiking computation. However, some foundation SNN backbones (including Spikformer and SEW ResNet) suffer from non-spike computations (integer-float multiplications) caused by the structure of their residual connections. These non-spike computations increase SNNs'power consumption and make them unsuitable for deployment on mainstream neuromorphic hardware. In this paper, we analyze the spike-driven behavior of the residual connection methods in SNNs. We then present Spikingformer, a novel spiking transformer backbone that merges the MS Residual connection with Self-Attention in a biologically plausible way to address the non-spike computation challenge in Spikformer while maintaining global modeling capabilities. We evaluate Spikingformer across 13 datasets spanning large static images, neuromorphic data, and natural language tasks, and demonstrate the effectiveness and universality of Spikingformer, setting a vital benchmark for spiking neural networks. In addition, with the spike-driven features and global modeling capabilities, Spikingformer is expected to become a more efficient general-purpose SNN backbone towards energy-efficient artificial intelligence. Code: https://github.com/TheBrainLab/Spikingformer
