DualSpecSE: A Dual-Path Speech Enhancement Network Integrating Mel and Complex Spectrograms

2026-09-12

AI Summary

本文提出DualSpecSE，通过结合Mel谱图和复数谱图的双路径架构来提升语音增强效果，提高自动语音识别性能和语音重建质量。

Abstract

In this paper, we propose DualSpecSE, a speech enhancement framework that jointly models Mel-spectrogram and complex spectrogram in a dual-path architecture for improved ASR performance and higher-quality speech reconstruction. The Mel branch learns coarse-grained acoustic representations and produces enhanced Mel-spectrograms for direct ASR usage, while the complex branch refines fine-grained spectral details for high-fidelity waveform reconstruction. Built upon the cross-band and narrow-band blocks from CleanMel, DualSpecSE introduces an interaction module and a fusion module to enable effective information exchange between the two branches. The model simultaneously outputs enhanced Mel and complex spectrogram without requiring a pretrained vocoder. Experimental results demonstrate consistent improvements in speech fidelity, perceptual quality, and ASR performance. Codes and audio samples are available.

Problem

• speech enhancement
• ASR performance
• speech reconstruction
• Mel-spectrogram
• complex spectrogram

Innovation

• Dual-Path Architecture
• Interaction Module
• Fusion Module
• Mel-spectrogram and Complex Spectrogram