Exploring the Potentials of Spiking Neural Networks for Image Deraining

2025-12-01

AI Summary

To address the weak spatial contextual modeling and insufficient high-frequency detail representation of spiking neural networks (SNNs) in image deraining, this paper proposes the Visual Leaky Integrate-and-Fire (LIF) neuron, which overcomes the frequency-domain saturation and limited spatial perception of conventional LIF units. Furthermore, we design a hierarchical multi-scale SNN framework incorporating a spike-based decomposition enhancement module and a lightweight spiking multi-scale unit to strengthen low-level visual feature representation. Evaluated on five standard deraining benchmarks, our method consistently outperforms existing SNN-based approaches, achieving average improvements of 2.1 dB in PSNR and 0.018 in SSIM. Notably, it consumes only 13% of the energy required by comparable non-spiking methods. This work constitutes the first systematic demonstration of SNNs’ feasibility and superiority for efficient, high-performance image restoration.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Computer Vision: Representation Learning for Vision
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Biologically plausible and energy-efficient frameworks such as Spiking Neural Networks (SNNs) have not been sufficiently explored in low-level vision tasks. Taking image deraining as an example, this study addresses the representation of the inherent high-pass characteristics of spiking neurons, specifically in image deraining and innovatively proposes the Visual LIF (VLIF) neuron, overcoming the obstacle of lacking spatial contextual understanding present in traditional spiking neurons. To tackle the limitation of frequency-domain saturation inherent in conventional spiking neurons, we leverage the proposed VLIF to introduce the Spiking Decomposition and Enhancement Module and the lightweight Spiking Multi-scale Unit for hierarchical multi-scale representation learning. Extensive experiments across five benchmark deraining datasets demonstrate that our approach significantly outperforms state-of-the-art SNN-based deraining methods, achieving this superior performance with only 13% of their energy consumption. These findings establish a solid foundation for deploying SNNs in high-performance, energy-efficient low-level vision tasks.

Problem

• Develops VLIF neuron for spatial context in image deraining
• Introduces modules for hierarchical multi-scale representation learning
• Enhances SNN performance and energy efficiency in low-level vision

Innovation

• VLIF neuron enhances spatial contextual understanding
• Spiking modules enable hierarchical multi-scale representation learning
• Achieves superior performance with 13% energy consumption