Unsupervised Blind Joint Dereverberation and Room Acoustics Estimation with Diffusion Models

2024-08-14

arXiv.org

AI Summary

This work addresses the joint problem of monaural blind dereverberation and room acoustic parameter estimation. We propose BUDDy, an unsupervised Bayesian framework that integrates measurement-fidelity likelihood with an unconditional speech diffusion prior via posterior sampling. BUDDy iteratively optimizes both a parametric band-limited exponential-decay RIR model and clean speech through reverse-diffusion trajectories. To our knowledge, this is the first method to incorporate diffusion models into unsupervised blind RIR estimation and dereverberation, eliminating the need for supervised training and enabling cross-scenario adaptability. Experiments demonstrate significant improvements over unsupervised baselines under diverse reverberation conditions; successful application to high-resolution singing voice dereverberation; accurate RIR estimation; and substantial gains in perceptual naturalness and intelligibility, confirmed by subjective evaluation. The core innovation lies in a diffusion-prior-driven unsupervised joint inversion framework for blind acoustic modeling and speech enhancement.

Technology Category

• Machine Learning: Unsupervised & Self-Supervised Learning
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Diffusion Models for Vision

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This paper presents an unsupervised method for single-channel blind dereverberation and room impulse response (RIR) estimation, called BUDDy. The algorithm is rooted in Bayesian posterior sampling: it combines a likelihood model enforcing fidelity to the reverberant measurement, and an anechoic speech prior implemented by an unconditional diffusion model. We design a parametric filter representing the RIR, with exponential decay for each frequency subband. Room acoustics estimation and speech dereverberation are jointly carried out, as the filter parameters are iteratively estimated and the speech utterance refined along the reverse diffusion trajectory. In a blind scenario where the room impulse response is unknown, BUDDy successfully performs speech dereverberation in various acoustic scenarios, significantly outperforming other blind unsupervised baselines. Unlike supervised methods, which often struggle to generalize, BUDDy seamlessly adapts to different acoustic conditions. This paper extends our previous work by offering new experimental results and insights into the algorithm's performance and versatility. We first investigate the robustness of informed dereverberation methods to RIR estimation errors, to motivate the joint acoustic estimation and dereverberation paradigm. Then, we demonstrate the adaptability of our method to high-resolution singing voice dereverberation, study its performance in RIR estimation, and conduct subjective evaluation experiments to validate the perceptual quality of the results, among other contributions. Audio samples and code can be found online.

Problem

• Unsupervised blind dereverberation and RIR estimation
• Joint room acoustics estimation and speech dereverberation
• Adapting to diverse acoustic conditions without supervision

Innovation

• Unsupervised blind joint dereverberation and RIR estimation
• Bayesian posterior sampling with diffusion models
• Parametric filter for frequency subband decay