Convergence Of Consistency Model With Multistep Sampling Under General Data Assumptions

2025-05-06

AI Summary

This work studies the convergence of multi-step sampling in consistency models under realistic data distribution assumptions—namely, bounded support or rapidly decaying tails, coupled with mild smoothness. We propose a perturbation smoothing technique that, for the first time under weak data assumptions, establishes simultaneous convergence guarantees in both Wasserstein and total variation (TV) distances. Our method integrates generalized forward process modeling, the consistency learning framework, and refined TV-distance estimation. Theoretically, we prove that multi-step sampling drives the generated distribution toward the target distribution at an explicit rate under approximate self-consistency. Experiments on two canonical forward processes demonstrate substantially improved sample quality over single-step sampling. The core contributions are: (i) the first rigorous TV-distance convergence theory for consistency models; and (ii) an extension of convergence analysis to more realistic, non-Gaussian, and potentially heavy-tailed data distributions.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

Diffusion models accomplish remarkable success in data generation tasks across various domains. However, the iterative sampling process is computationally expensive. Consistency models are proposed to learn consistency functions to map from noise to data directly, which allows one-step fast data generation and multistep sampling to improve sample quality. In this paper, we study the convergence of consistency models when the self-consistency property holds approximately under the training distribution. Our analysis requires only mild data assumption and applies to a family of forward processes. When the target data distribution has bounded support or has tails that decay sufficiently fast, we show that the samples generated by the consistency model are close to the target distribution in Wasserstein distance; when the target distribution satisfies some smoothness assumption, we show that with an additional perturbation step for smoothing, the generated samples are close to the target distribution in total variation distance. We provide two case studies with commonly chosen forward processes to demonstrate the benefit of multistep sampling.

Problem

• Analyzing convergence of consistency models under approximate self-consistency
• Evaluating sample quality in Wasserstein and total variation distances
• Demonstrating benefits of multistep sampling for data generation

Innovation

• Consistency models enable one-step fast data generation
• Multistep sampling improves sample quality efficiently
• Mild data assumptions ensure broad applicability