Lapis: Laplacian Spiking Attention via First-Spike Timing and Membrane Leakage

2026-08-12

AI Summary

This work addresses the limitation of existing spiking vision Transformers, which adopt dense-network-style dot-product self-attention and thereby neglect the temporal information inherent in spike trains. To overcome this, the authors propose Lapis, a novel attention mechanism that directly incorporates first-spike timing into attention computation for the first time. Specifically, Lapis computes affinity via a Laplacian kernel applied to the L1 distance between first-spike latency vectors of queries and keys, aligning naturally with the membrane potential decay dynamics of leaky integrate-and-fire neurons and enabling multiplication-free, efficient attention. Combined with power-of-two rounding, row-wise normalization reduces to bit-shift operations. The method achieves 96.56% Top-1 accuracy on CIFAR-10 (only 0.53% lower than baseline) and reduces attention-path energy consumption by 14.5× on ImageNet-1K, attaining 83.25% accuracy with 3.28 mJ per image at 6-bit precision.

Abstract

Self-attention has become central to spiking vision transformers, yet its query-key scoring is still largely inherited from dense networks. Existing spiking variants either simplify dot product scoring or replace it with discrete operators, but spike timing, the native variable of a spiking network, does not directly define how tokens are related. We propose Lapis, a spiking attention mechanism that scores each token pair by the L1 distance between its query and key first-spike latency vectors under time-to-first-spike coding, and maps this distance to an affinity through a Laplacian kernel. The kernel's exponential decay matches the impulse response of a leaky integrate-and-fire membrane, so the accumulated latency difference determines the decay of a membrane trace, while row normalization reduces to a bit shift under power-of-two rounding. Scoring therefore needs only subtraction, absolute value, and accumulation, and removes all multiplication between query and key channels. Under a matched backbone and training schedule, Lapis reaches 96.56% top-1 accuracy on CIFAR-10, within 0.53 points of dot-product scoring. On ImageNet-1K, it reduces the estimated arithmetic energy of the attention path by 14.5x relative to dense dot-product attention. The deployed 6-bit model attains 83.25% top-1 accuracy at an estimated arithmetic energy of 3.28mJ per image.

Problem

• spiking attention
• self-attention
• spike timing
• query-key scoring
• spiking neural networks

Innovation

• spiking attention
• first-spike timing
• Laplacian kernel
• leaky integrate-and-fire
• energy-efficient computing