Adaptive Spiking Neurons for Vision and Language Modeling

2026-04-14

AI Summary

This work addresses the limitations of existing spiking neurons—namely insufficient performance, poor adaptability, and low training efficiency—in large-scale vision and language tasks. The authors propose Adaptive Spiking Neurons (ASN) and their normalized variant (NASN), the first neuron designs systematically guided by a functional perspective. By incorporating learnable membrane potential dynamics, an integer-based training–spiking inference paradigm, and normalization mechanisms, ASN significantly enhances training stability and model generalizability. Comprehensive experiments across five task categories and nineteen datasets spanning both vision and language domains demonstrate that the ASN family exhibits strong generalization capabilities and holds substantial promise as a universal spiking neuron model.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Neuro-Symbolic Learning
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Regarded as the third generation of neural networks, Spiking Neural Networks (SNNs) have garnered significant traction due to their biological plausibility and energy efficiency. Recent advancements in large models necessitate spiking neurons capable of high performance, adaptability, and training efficiency. In this work, we first propose a novel functional perspective that provides general guidance for designing the new generation of spiking neurons. Following the insightful guidelines, we propose the Adaptive Spiking Neuron (ASN), which incorporates trainable parameters to learn membrane potential dynamics and enable adaptive firing. ASN adopts an integer training and spike inference paradigm, facilitating efficient SNN training. To further enhance robustness, we propose a specialized variant of ASN, the Normalized Adaptive Spiking Neuron (NASN), which integrates normalization to stabilize training. We evaluate our neuron model on 19 datasets spanning five distinct tasks in both vision and language modalities, demonstrating the effectiveness and versatility of the ASN family. Our ASN family is expected to become the new generation of general-purpose spiking neurons.

Problem

• Spiking Neural Networks
• Adaptive Neurons
• Vision Modeling
• Language Modeling
• Training Efficiency

Innovation

• Spiking Neural Networks
• Adaptive Spiking Neuron
• Trainable Membrane Dynamics
• Integer Training
• Normalization