Class-frequency Guided Noise Schedule for Diffusion Models

2026-06-25

AI Summary

This work addresses the degradation in generation quality and diversity for low-frequency classes in diffusion models, which arises from biased score estimation due to data sparsity, while high-frequency classes dominate the score space and exacerbate class imbalance under long-tailed distributions. The study is the first to uncover the relationship between class frequency and multi-scale noise scheduling, proposing a class-frequency-guided noise scheduling strategy that assigns larger noise scales to low-frequency classes to improve their score estimation. Evaluated on long-tailed benchmarks such as CIFAR-100-LT and ImageNet-LT, the method significantly enhances generation quality and diversity across all classes, outperforming existing baselines in image generation, classification, and text-to-image tasks, thereby demonstrating the effectiveness and generalizability of frequency-aware noise scheduling.

Abstract

In this paper, we are the first to examine the correlations between class frequency and the multi-scale noise schedule within diffusion models. For score-based generative models, low-density regions often lead to inaccurately estimated scores, thereby compromising the generation quality. Although the multi-scale noise schedule can alleviate this issue during the diffusion process, low-frequency classes still face the challenge of large low-density regions, resulting in more inaccurate estimated scores than high-frequency classes. Furthermore, high-frequency classes tend to dominate the score space, causing a convergence of most data points towards generating samples from these classes. Consequently, samples generated within low-frequency classes exhibit suboptimal quality and limited diversity. To address this challenge, we propose the \textit{Class-frequency Guided (CFRG)} noise schedule, leveraging the insight that low-frequency classes should be endowed with larger-scale noises. To illustrate the effectiveness of our method, we conduct experiments on various tasks, including image generation, image classification, and text-to-image generation, using imbalanced datasets, \textit{i.e.}, CIFAR-100-LT, and ImageNet-LT. By employing the CFRG noise schedule, we achieve substantial improvements over baselines, manifesting the crucial role of frequency statistics in noise schedule design.

Problem

• class frequency
• diffusion models
• noise schedule
• score estimation
• imbalanced datasets

Innovation

• class-frequency guided
• diffusion models
• noise schedule
• imbalanced datasets
• score estimation