Data Warmup: Complexity-Aware Curricula for Efficient Diffusion Training

2026-04-08

AI Summary

Diffusion models often suffer from slow convergence during early training due to the absence of visual priors, particularly when handling highly complex data. To address this, this work proposes Data Warmup—a curriculum learning strategy that schedules training samples from simple to complex based on an offline-computed image complexity metric combining foreground ratio and semantic typicality. This approach introduces, for the first time, a semantic-aware complexity measure coupled with a temperature-annealed sampling mechanism, establishing a zero-overhead, composable curriculum paradigm that requires no modifications to the model architecture or loss function. When integrated with the SiT backbone on ImageNet at 256×256 resolution, the method achieves up to a 6.11 improvement in Inception Score and a 3.41 reduction in FID, while reaching baseline performance tens of thousands of training steps earlier. It also remains compatible with existing acceleration techniques such as REPA.

Abstract

A key inefficiency in diffusion training occurs when a randomly initialized network, lacking visual priors, encounters gradients from the full complexity spectrum--most of which it lacks the capacity to resolve. We propose Data Warmup, a curriculum strategy that schedules training images from simple to complex without modifying the model or loss. Each image is scored offline by a semantic-aware complexity metric combining foreground dominance (how much of the image salient objects occupy) and foreground typicality (how closely the salient content matches learned visual prototypes). A temperature-controlled sampler then prioritizes low-complexity images early and anneals toward uniform sampling. On ImageNet 256x256 with SiT backbones (S/2 to XL/2), Data Warmup improves IS by up to 6.11 and FID by up to 3.41, reaching baseline quality tens of thousands of iterations earlier. Reversing the curriculum (exposing hard images first) degrades performance below the uniform baseline, confirming that the simple-to-complex ordering itself drives the gains. The method combines with orthogonal accelerators such as REPA and requires only ~10 minutes of one-time preprocessing with zero per-iteration overhead.

Problem

• diffusion training
• training inefficiency
• complexity spectrum
• visual priors
• curriculum learning

Innovation

• Data Warmup
• curriculum learning
• diffusion training
• complexity-aware sampling
• semantic-aware complexity