diffusion models

This work addresses speech enhancement and dereverberation via a diffusion generative model based on stochastic differential equations (SDEs). Methodologically, it (1) models the forward process as the realistic speech degradation trajectory—rather than idealized Gaussian noise addition—thereby better aligning with physical degradation mechanisms; (2) initializes reverse sampling with mixed noise (not pure Gaussian noise), substantially accelerating convergence; and (3) unifies enhancement and dereverberation within a single conditional U-Net framework, supporting diverse samplers (e.g., DDIM, Euler). Experiments demonstrate that high-fidelity clean speech reconstruction is achieved in only 30 sampling steps, yielding significant computational efficiency gains. The model exhibits superior cross-dataset generalization compared to state-of-the-art discriminative models, and achieves top performance on both real-world recordings and subjective listening evaluations.

To address the low denoising efficiency and reconstruction quality degradation caused by sequential randomness in discrete diffusion models, this paper proposes a “planning-based denoising” framework that decouples the process into a learnable position planner and a local denoiser, enabling on-demand identification of critical positions and precise token/image restoration. This two-stage paradigm overcomes inherent limitations of conventional mask diffusion—namely, uniform or random denoising—by introducing iterative adaptive mask selection and joint token/image training. Evaluated on text8, OpenWebText, and ImageNet 256×256, our method significantly outperforms state-of-the-art mask diffusion models: it achieves language modeling perplexity closely approaching autoregressive baselines while simultaneously improving both inference efficiency and fidelity in image generation.

The relative difficulty of denoising tasks across timesteps in diffusion models remains controversial. Method: This work systematically quantifies denoising difficulty per timestep, leveraging both the convergence behavior of denoising error and the relative entropy between true and predicted distributions—revealing that early (low-timestep) denoising is significantly more challenging. Building on this insight, we propose a “curriculum learning” paradigm: timesteps are clustered by difficulty and trained progressively in stages, with joint optimization of the noise schedule. Contribution/Results: Our approach departs from conventional parallel full-timestep training, requiring no architectural or loss-function modifications and remaining compatible with diverse diffusion model enhancements. Extensive experiments on unconditional generation, class-conditional generation, and text-to-image synthesis demonstrate substantial improvements in both model performance and convergence speed.

To address the unreliable sample generation by diffusion models in out-of-distribution (OOD) regions, this paper proposes Gen-neG—a novel framework that, for the first time, integrates an oracle prior capable of identifying OOD samples into the DDPM sampling process. Without requiring labeled in-distribution data, Gen-neG explicitly steers denoising trajectories away from regions outside the data support via oracle-guided negative-sample constraints. Furthermore, it incorporates GAN-inspired discrimination by introducing an oracle-assisted discriminator guidance mechanism, unifying discriminative supervision with diffusion-based generation. Evaluated on autonomous driving collision avoidance simulation and safe human motion generation, Gen-neG significantly reduces OOD generation rates—by an average of 38.7%—while improving physical plausibility and safety of generated samples. This work establishes a new paradigm for safety-aware generative modeling grounded in reliable domain priors.

To address the degradation in generation quality of DDIM sampling under low-step regimes, this work proposes replacing the standard Gaussian backward transition kernel with a moment-matching Gaussian Mixture Model (GMM), constrained solely by first- and second-order central moments for efficient distribution approximation. This is the first integration of moment-matching GMM into the DDIM framework—requiring no additional network parameters or extra training overhead. On ImageNet 256×256, our method achieves an FID of 6.94 (a 3.21 improvement) and an Inception Score of 207.85 (a 11.12 gain) using only 10 sampling steps, significantly outperforming the DDIM baseline. The approach combines theoretical elegance—rooted in moment-based distribution matching—with practical efficiency, offering a novel paradigm for accelerating diffusion model inference without compromising sample quality.

Existing theoretical analyses of diffusion generative models are fragmented and suffer from inconsistent notation, hindering unified understanding and principled development. Method: This work establishes a rigorous, unified mathematical framework grounded in fundamental properties of Gaussian distributions. It systematically derives the closed-form marginal distribution of the forward noising process, the analytical form of the reverse posterior, and the variational lower bound, ultimately yielding an optimization objective equivalent to noise prediction. Contribution/Results: The framework reveals the intrinsic equivalence between DDIM and rectified flow; provides a unified probabilistic interpretation of classifier-guided and classifier-free guidance; and integrates SDE/ODE formulations, the Fokker–Planck equation, flow matching, and multi-scale modeling—ensuring both theoretical coherence and practical implementability. Validated on mainstream models including Stable Diffusion, the framework enables efficient sampling and precise modeling while unifying disparate theoretical perspectives.

Standard classifier-guided diffusion models tend to generate samples predominantly from high-density regions of the data distribution, struggling to capture rare modes in the tails of class-conditional distributions. This work proposes a density-aware guidance approach that operates solely during sampling, requiring no additional training or architectural modifications. The method employs a dual guidance mechanism: it modulates classifier gradients to encourage exploration of low-confidence (low-density) regions while simultaneously constraining the generation trajectory to remain close to the true data manifold. Leveraging a pretrained ADM and an ImageNet classifier, the approach significantly improves recall at 64×64 resolution without degrading FID, and at 256×256 resolution achieves both high perceptual quality and diversity, effectively balancing the generation of rare samples with overall fidelity.

This work addresses the inefficiency of traditional classifier-guided diffusion models, which require separate training of a classifier and a generative model, leading to redundancy and high computational costs. To overcome this, the authors propose an efficient approach for conditional speech generation that repurposes a pretrained speech classifier as a shared backbone network. By freezing the backbone’s parameters and training only lightweight auxiliary subnetworks, the method enables conditional generation while unifying discriminative modeling and speech synthesis within a single architecture for the first time. The framework integrates noise-conditional classification, log-Mel spectrogram space modeling, and denoising score matching, achieving high-quality speech synthesis with substantially reduced memory and computational overhead.

This paper identifies and formalizes the pervasive “noise shift” problem in diffusion model sampling: a systematic mismatch between the prescribed noise schedule and the actual noise levels of intermediate latent states, leading to inaccurate denoising updates and degraded out-of-distribution generalization. To address this, we propose Noise-Aware Guidance—a classifier-free method that explicitly corrects sampling trajectories via a noise-conditioned dropout training mechanism, without requiring external classifiers. Our approach integrates empirical noise estimation with dynamic adjustment, jointly optimizing generation within the reverse SDE/ODE framework. Evaluated on ImageNet synthesis and diverse fine-tuning tasks, the method achieves significant improvements in FID, LPIPS, and other metrics, effectively mitigating noise shift while enhancing sampling stability and generalization capability.

Traditional discrete diffusion models suffer from low generation efficiency due to their reliance on a fixed forward noising process and a factorized reverse process, which hinders accurate approximation of the target distribution within a limited number of sampling steps. This work proposes Forward-Learned Discrete Diffusion (FLDD), which introduces, for the first time in discrete diffusion, a learnable non-Markovian forward process. By end-to-end optimizing learnable marginal and posterior distributions, FLDD substantially reduces the gap between the model and the target distribution while preserving the factorized structure of the reverse process. Built upon a variational inference framework, FLDD consistently outperforms existing methods across multiple benchmark tasks under the same sampling budget.