Learning When to Denoise: Optimizing Asynchronous Schedules for Latent Diffusion

2026-06-17

AI Summary

This work addresses the critical yet underexplored challenge of optimizing asynchronous denoising schedules in multi-representation diffusion models, where such scheduling significantly impacts generation quality but lacks efficient optimization strategies. The authors propose a learnable class of convex monotonic parameterized schedules and introduce a schedule-correction objective that enables joint optimization of the denoising trajectory and the generative model with negligible additional training cost. Integrating asynchronous flow matching, multi-representation modeling, and an AutoGuidance mechanism, the method achieves FID scores of 1.05 (with guidance) and 2.37 (without guidance) on ImageNet 256×256 after only 200 training epochs—surpassing existing 800-epoch baselines with just one-quarter of the training effort. Further training to 600 epochs yields state-of-the-art results of FID 1.02 and 2.14, substantially advancing both efficiency and sample quality.

Abstract

Multi-representation diffusion models can improve visual synthesis by denoising complementary views of an image, but their performance depends critically on the asynchronous schedule that determines when each representation is denoised. We propose to learn this schedule. Our method formulates asynchronous flow matching over multiple representation spaces and uses a schedule-corrected objective that keeps each representation's local noising-time weights fixed as the schedule changes. We instantiate the schedule with a flexible parametric class that is convex and monotone by construction, and learn it using a fast joint probe with less than 1% additional training compute. On ImageNet 256x256, the learned schedule substantially improves both convergence speed and final quality under a matched 675M-parameter XL backbone. With AutoGuidance, our 200-epoch model reaches FID 1.05, matching the 800-epoch SFD-XL baseline with 4x less training. Training to 600 epochs further improves to FID 1.02, outperforming the 1B-parameter SFD-XXL result of FID 1.04 while using a smaller model. In the unguided setting, our 200-epoch model reaches FID 2.37, already below the best 800-epoch SFD-XL result (2.54) at 4x less training, and improves to FID 2.14 at 600 epochs. Code is available at https://github.com/bsq532087/LWD

Problem

• asynchronous schedule
• latent diffusion
• denoising
• multi-representation
• diffusion models

Innovation

• asynchronous scheduling
• latent diffusion
• multi-representation diffusion
• flow matching
• schedule learning