Neuromorphic Speech Enhancement with Dual-Branch Spiking Neural Networks

2026-06-22

AI Summary

This work addresses the performance gap between spiking neural networks (SNNs) and artificial neural networks (ANNs) in speech enhancement, where existing SNN approaches are hindered by binary activations and overly simplistic architectures. To overcome these limitations, the authors propose GSU-DBNet, a novel dual-branch SNN that introduces gated spiking units (GSUs) and dual-path time-frequency modeling to jointly optimize magnitude and complex spectral mask estimation. This architecture substantially enhances feature representation capabilities while maintaining extreme parameter efficiency—requiring only 394K parameters, equivalent to 4.5%–10.6% of typical ANN sizes. Evaluated on standard datasets, GSU-DBNet achieves a PESQ score of 3.04, significantly outperforming current SNN-based methods and establishing a new state of the art in both performance and parameter efficiency for SNN speech enhancement.

Abstract

Spiking neural network (SNN)-based neuromorphic speech enhancement has emerged as a promising paradigm due to its energy efficiency, yet it still underperforms classical artificial neural network (ANN)-based approaches owing to binary activations and the lack of well-designed network architectures. To overcome this limitation, we propose a novel dual-branch spiking neural network architecture equipped with a gated spiking unit (GSU), termed GSU-DBNet. Specifically, GSU-DBNet simultaneously models the speech magnitude spectrum and complex spectrum, predicting the corresponding magnitude and complex spectral masks. Meanwhile, a dual-path GSU module is adopted to exploit temporal and frequency information for enhanced spatiotemporal feature representation. Experiments on a popular benchmark dataset show that GSU-DBNet achieves a PESQ score of 3.04 with only 394K parameters, outperforming existing SNN-based methods while using only 4.5%--10.6% of the parameters of representative ANN-based models.

Problem

• spiking neural network
• speech enhancement
• neuromorphic computing
• binary activations
• network architecture

Innovation

• Spiking Neural Networks
• Neuromorphic Speech Enhancement
• Dual-Branch Architecture
• Gated Spiking Unit
• Spectral Masking