GDiffuSE: Diffusion-based speech enhancement with noise model guidance

2025-10-05

AI Summary

To address the limited robustness of speech enhancement under unseen noise conditions, this paper proposes a novel denoising diffusion probabilistic model (DDPM)-based approach. The method introduces two key innovations: (1) a lightweight auxiliary model for estimating noise distribution, and (2) a noise-aware conditional guidance mechanism—first of its kind—that seamlessly incorporates the estimated noise prior into the diffusion denoising process. Additionally, it enables effective transfer of large-scale pre-trained speech generation diffusion models to the enhancement task. Evaluated on LibriSpeech and BBC Sound Effects datasets, the method demonstrates significant performance gains over current state-of-the-art approaches under noise-mismatch conditions. Results confirm its strong generalization capability and practical deployability in real-world scenarios.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Speech
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Security and Privacy: Privacy-enhancing technologies

Abstract

This paper introduces a novel speech enhancement (SE) approach based on a denoising diffusion probabilistic model (DDPM), termed Guided diffusion for speech enhancement (GDiffuSE). In contrast to conventional methods that directly map noisy speech to clean speech, our method employs a lightweight helper model to estimate the noise distribution, which is then incorporated into the diffusion denoising process via a guidance mechanism. This design improves robustness by enabling seamless adaptation to unseen noise types and by leveraging large-scale DDPMs originally trained for speech generation in the context of SE. We evaluate our approach on noisy signals obtained by adding noise samples from the BBC sound effects database to LibriSpeech utterances, showing consistent improvements over state-of-the-art baselines under mismatched noise conditions. Examples are available at our project webpage.

Problem

• Develops diffusion-based speech enhancement with noise guidance
• Improves robustness for unseen noise types using generative models
• Enhances noisy speech by estimating noise distribution during denoising

Innovation

• Denoising diffusion probabilistic model for speech enhancement
• Lightweight helper model estimates noise distribution
• Guidance mechanism adapts to unseen noise types