QP-SNN: Quantized and Pruned Spiking Neural Networks

2025-02-09

AI Summary

To address the conflicting challenges of large model size, high resource overhead, and low energy efficiency in deploying Spiking Neural Networks (SNNs) on edge devices, this paper proposes a lightweight, hardware-friendly co-optimization framework for SNNs. Methodologically, it introduces a novel weight rescaling strategy to enhance representational capacity under low-bit quantization and designs a structured pruning criterion based on spatiotemporal spike activity singular values to precisely remove redundant convolutional kernels. The framework preserves the event-driven nature of SNNs while significantly reducing memory footprint and computational cost. Experiments across multiple benchmark datasets demonstrate state-of-the-art accuracy–efficiency trade-offs on edge platforms. The proposed approach establishes a new paradigm for high-performance SNN deployment under severe resource constraints.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Learning on the Edge & Model Compression
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Brain-inspired Spiking Neural Networks (SNNs) leverage sparse spikes to encode information and operate in an asynchronous event-driven manner, offering a highly energy-efficient paradigm for machine intelligence. However, the current SNN community focuses primarily on performance improvement by developing large-scale models, which limits the applicability of SNNs in resource-limited edge devices. In this paper, we propose a hardware-friendly and lightweight SNN, aimed at effectively deploying high-performance SNN in resource-limited scenarios. Specifically, we first develop a baseline model that integrates uniform quantization and structured pruning, called QP-SNN baseline. While this baseline significantly reduces storage demands and computational costs, it suffers from performance decline. To address this, we conduct an in-depth analysis of the challenges in quantization and pruning that lead to performance degradation and propose solutions to enhance the baseline's performance. For weight quantization, we propose a weight rescaling strategy that utilizes bit width more effectively to enhance the model's representation capability. For structured pruning, we propose a novel pruning criterion using the singular value of spatiotemporal spike activities to enable more accurate removal of redundant kernels. Extensive experiments demonstrate that integrating two proposed methods into the baseline allows QP-SNN to achieve state-of-the-art performance and efficiency, underscoring its potential for enhancing SNN deployment in edge intelligence computing.

Problem

• Optimizing SNNs for edge devices
• Reducing SNN storage and computational costs
• Enhancing SNN performance with quantization and pruning

Innovation

• Quantized and Pruned SNN
• Weight Rescaling Strategy
• Singular Value Pruning Criterion