EDNet: A Distortion-Agnostic Speech Enhancement Framework with Gating Mamba Mechanism and Phase Shift-Invariant Training

2025-06-19

AI Summary

Real-world speech is often degraded by single- or multi-type distortions—including noise, reverberation, and bandwidth limitation—yet existing enhancement methods rely on strong prior assumptions or fixed paradigms, resulting in poor generalization under out-of-distribution degradations. To address this, we propose a robust, distortion-agnostic speech enhancement framework. Our method introduces a novel learnable gated Mamba module that dynamically orchestrates complementary “erasure” (suppression) and “drawing” (reconstruction) strategies. We further propose phase-shift-invariant training (PSIT), integrating complex-valued spectral modeling with dynamic phase-alignment supervision. The entire framework enables end-to-end joint time-frequency optimization. Extensive experiments demonstrate state-of-the-art performance across denoising, dereverberation, bandwidth extension, and multi-distortion joint enhancement tasks. Crucially, our approach achieves significantly improved cross-distortion generalization without requiring distortion-type priors.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Natural Language Processing: Speech
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Speech signals in real-world environments are frequently affected by various distortions such as additive noise, reverberation, and bandwidth limitation, which may appear individually or in combination. Traditional speech enhancement methods typically rely on either masking, which focuses on suppressing non-speech components while preserving observable structure, or mapping, which seeks to recover clean speech through direct transformation of the input. Each approach offers strengths in specific scenarios but may be less effective outside its target conditions. We propose the Erase and Draw Network (EDNet), a distortion-agnostic speech enhancement framework designed to handle a broad range of distortion types without prior assumptions about task or input characteristics. EDNet consists of two main components: (1) the Gating Mamba (GM) module, which adaptively combines masking and mapping through a learnable gating mechanism that selects between suppression (Erase) and reconstruction (Draw) based on local signal features, and (2) Phase Shift-Invariant Training (PSIT), a shift tolerant supervision strategy that improves phase estimation by enabling dynamic alignment during training while remaining compatible with standard loss functions. Experimental results on denoising, dereverberation, bandwidth extension, and multi distortion enhancement tasks show that EDNet consistently achieves strong performance across conditions, demonstrating its architectural flexibility and adaptability to diverse task settings.

Problem

• Handles diverse speech distortions like noise and reverberation
• Combines masking and mapping via adaptive gating mechanism
• Improves phase estimation with shift-invariant training

Innovation

• Gating Mamba combines masking and mapping adaptively
• Phase Shift-Invariant Training improves phase estimation
• Distortion-agnostic framework handles diverse speech distortions