A Fourier Space Perspective on Diffusion Models

2025-05-16

AI Summary

Diffusion models exhibit a Fourier-domain inductive bias in the standard DDPM forward process: high-frequency components suffer excessively rapid SNR decay, causing the reverse process to violate the Gaussianity assumption and degrading high-frequency detail synthesis. To address this, we propose an isochronous frequency-domain noising scheme, wherein all frequency components are progressively corrupted at identical rates—eliminating the inherent frequency-dependent hierarchy in conventional diffusion. This design fundamentally redefines diffusion dynamics modeling from the Fourier perspective, requiring no modifications to the reverse sampler. Experiments demonstrate substantial improvements in generation quality for high-frequency–dominant tasks sensitive to edges and textures, while maintaining performance parity with DDPM on standard image benchmarks including CIFAR-10 and CelebA.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Diffusion models are state-of-the-art generative models on data modalities such as images, audio, proteins and materials. These modalities share the property of exponentially decaying variance and magnitude in the Fourier domain. Under the standard Denoising Diffusion Probabilistic Models (DDPM) forward process of additive white noise, this property results in high-frequency components being corrupted faster and earlier in terms of their Signal-to-Noise Ratio (SNR) than low-frequency ones. The reverse process then generates low-frequency information before high-frequency details. In this work, we study the inductive bias of the forward process of diffusion models in Fourier space. We theoretically analyse and empirically demonstrate that the faster noising of high-frequency components in DDPM results in violations of the normality assumption in the reverse process. Our experiments show that this leads to degraded generation quality of high-frequency components. We then study an alternate forward process in Fourier space which corrupts all frequencies at the same rate, removing the typical frequency hierarchy during generation, and demonstrate marked performance improvements on datasets where high frequencies are primary, while performing on par with DDPM on standard imaging benchmarks.

Problem

• Analyzes frequency bias in diffusion models' forward process
• Identifies high-frequency corruption issue in standard DDPM
• Proposes equal-rate frequency corruption for improved generation

Innovation

• Analyzes diffusion models in Fourier space
• Proposes equal-frequency corruption forward process
• Improves high-frequency generation quality