🤖 AI Summary
Diffusion model sampling typically requires thousands of steps; existing acceleration methods often suffer significant degradation in generation quality due to timestep misalignment—i.e., mismatch between discrete integration steps and the underlying continuous-time distribution. This work is the first to formally model diffusion sampling as a numerical integration process, identifying timestep misalignment as the primary cause of performance collapse under few-step sampling. To address this, we propose Timestep Aligner: a lightweight, plug-and-play module that dynamically remaps timesteps via distribution alignment and reparameterizes conditional inputs—requiring no model retraining. The method is compatible with mainstream samplers including DDIM and DEIS. On LSUN Bedroom, it reduces FID from 9.65 to 6.07 using only 10 sampling steps, markedly improving both image fidelity and diversity. Our approach achieves high efficiency and strong generalization across architectures and datasets.
📝 Abstract
A diffusion model, which is formulated to produce an image using thousands of denoising steps, usually suffers from a slow inference speed. Existing acceleration algorithms simplify the sampling by skipping most steps yet exhibit considerable performance degradation. By viewing the generation of diffusion models as a discretized integrating process, we argue that the quality drop is partly caused by applying an inaccurate integral direction to a timestep interval. To rectify this issue, we propose a timestep aligner that helps find a more accurate integral direction for a particular interval at the minimum cost. Specifically, at each denoising step, we replace the original parameterization by conditioning the network on a new timestep, which is obtained by aligning the sampling distribution to the real distribution. Extensive experiments show that our plug-in design can be trained efficiently and boost the inference performance of various state-of-the-art acceleration methods, especially when there are few denoising steps. For example, when using 10 denoising steps on the popular LSUN Bedroom dataset, we improve the FID of DDIM from 9.65 to 6.07, simply by adopting our method for a more appropriate set of timesteps. Code will be made publicly available.
