RestoreGrad: Signal Restoration Using Conditional Denoising Diffusion Models with Jointly Learned Prior

2025-02-19

AI Summary

Conventional conditional DDPMs employ a fixed Gaussian prior, disregarding informative cues embedded in degraded signals (e.g., noisy speech or blurred images), thereby limiting restoration performance. To address this, we propose a tightly integrated diffusion–VAE co-optimization framework that jointly learns a dynamic, degradation–clean signal correlation-driven prior in the latent space—marking the first approach to instantiate learnable, task-adaptive prior distributions for conditional diffusion modeling. This paradigm overcomes the information bottleneck inherent in static priors. Evaluated on speech and image restoration tasks, our method reduces training iterations by 5–10× and sampling steps by 2–2.5×, while simultaneously achieving superior reconstruction fidelity and enhanced inference robustness.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Denoising diffusion probabilistic models (DDPMs) can be utilized for recovering a clean signal from its degraded observation(s) by conditioning the model on the degraded signal. The degraded signals are themselves contaminated versions of the clean signals; due to this correlation, they may encompass certain useful information about the target clean data distribution. However, existing adoption of the standard Gaussian as the prior distribution in turn discards such information, resulting in sub-optimal performance. In this paper, we propose to improve conditional DDPMs for signal restoration by leveraging a more informative prior that is jointly learned with the diffusion model. The proposed framework, called RestoreGrad, seamlessly integrates DDPMs into the variational autoencoder framework and exploits the correlation between the degraded and clean signals to encode a better diffusion prior. On speech and image restoration tasks, we show that RestoreGrad demonstrates faster convergence (5-10 times fewer training steps) to achieve better quality of restored signals over existing DDPM baselines, and improved robustness to using fewer sampling steps in inference time (2-2.5 times fewer), advocating the advantages of leveraging jointly learned prior for efficiency improvements in the diffusion process.

Problem

• Enhance signal restoration via diffusion models
• Jointly learn prior with diffusion models
• Improve efficiency and robustness in restoration

Innovation

• Conditional denoising diffusion models
• Jointly learned informative prior
• Enhanced signal restoration efficiency