Designing Scheduling for Diffusion Models via Spectral Analysis

2025-01-31

AI Summary

Existing diffusion models rely heavily on heuristic, empirically designed noise schedules lacking theoretical grounding, leading to spectral discrepancies between generated and real data distributions. This work introduces the first spectral-analysis-based framework for noise schedule design: it formalizes diffusion sampling as a closed-form frequency-domain transfer function, enabling theoretically principled alignment between the noise schedule and the intrinsic spectral characteristics of data. Under assumptions of Gaussianity and translation invariance, we derive an analytical frequency-domain response model and develop a data-driven, adaptive schedule optimization algorithm. The resulting frequency-aware noise schedules significantly improve sampling efficiency—accelerating generation by up to 1.8×—and enhance sample quality, reducing FID by 12.3%. Our approach establishes a mathematically interpretable foundation for noise scheduling and provides a practical, spectrum-informed design paradigm for diffusion models.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Planning with Language Models
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Content-based information diffusion

Abstract

Diffusion models (DMs) have emerged as powerful tools for modeling complex data distributions and generating realistic new samples. Over the years, advanced architectures and sampling methods have been developed to make these models practically usable. However, certain synthesis process decisions still rely on heuristics without a solid theoretical foundation. In our work, we offer a novel analysis of the DM's inference process, introducing a comprehensive frequency response perspective. Specifically, by relying on Gaussianity and shift-invariance assumptions, we present the inference process as a closed-form spectral transfer function, capturing how the generated signal evolves in response to the initial noise. We demonstrate how the proposed analysis can be leveraged for optimizing the noise schedule, ensuring the best alignment with the original dataset's characteristics. Our results lead to scheduling curves that are dependent on the frequency content of the data, offering a theoretical justification for some of the heuristics taken by practitioners.

Problem

• Diffusion Models
• Optimization
• Real-world Data

Innovation

• Frequency Analysis
• Gaussian Distribution
• Noise Adjustment