Generalization Bounds of Spiking Neural Networks via Rademacher Complexity

2026-04-26

AI Summary

Existing theoretical frameworks lack a clear characterization of the generalization capability of spiking neural networks (SNNs), particularly offering no performance guarantees on unseen data. This work establishes, for the first time, tight generalization upper bounds for SNNs under various integrate-and-fire mechanisms using Rademacher complexity. The derived bounds quantitatively reveal dependencies on network depth (exponential), width (superlinear but subquadratic), parameter norm (polynomial), and sample size (inverse linear), and crucially demonstrate that these bounds are independent of the internal neuronal computations. By bridging a significant gap in the theoretical understanding of SNNs, this study not only advances foundational analysis but also provides principled guidance for designing efficient and reliable SNN architectures.

Abstract

Spiking Neural Networks (SNNs) have garnered increasing attention as one of bio-inspired models due to their great potential in neuromorphic computing and sparse computation. Many practical algorithms and techniques have been developed; however, theoretical understandings of the generalization, that is, the extent to which SNNs perform well on unseen data, are far from clear. Recent advances disclosed an excitation-dependent and architecture-related generalization bound such that the Rademacher complexity of SNNs with stochastic firing can be upper bounded by an exponential function relative to the excitation probability and the architecture depth. In this paper, we theoretically investigate the generalization bounds of SNNs with several integration-and-fire schemes via Rademacher complexity. We recognize that the empirical Rademacher complexity of SNNs is close to the SNN configurations, which is exponential to the network depth and the maximum time duration of received spike sequences, superlinear and subquadratic to the network width, polynomial to the parameter norm, inverse-linear to the number of training samples, and independent of the computations within spiking neurons, achieving a more precise rate than conventional studies. Our theoretical results may support the scope of SNN theories and shed some insight into the development of SNNs.

Problem

• Spiking Neural Networks
• Generalization Bounds
• Rademacher Complexity
• Theoretical Understanding
• Neuromorphic Computing

Innovation

• Spiking Neural Networks
• Generalization Bounds
• Rademacher Complexity
• Integration-and-Fire Models
• Neuromorphic Computing