Cross-fluctuation phase transitions reveal sampling dynamics in diffusion models

2025-10-31

AI Summary

This work investigates phase transition mechanisms in the sampling dynamics of score-based diffusion models. Specifically, it addresses the phenomenon wherein samples gradually reconstruct the target distribution from an isotropic Gaussian during reverse-time sampling. To detect such transitions, we propose an *n*-th-order cross-fluctuation–based phase transition detection method. This is the first approach to extend Markov chain coupling and mixing concepts to continuous diffusion processes, thereby unifying analytical frameworks for both discrete and continuous generative dynamics. Leveraging the closed-form solution of variance-preserving stochastic differential equations (SDEs), our method efficiently computes cross-fluctuations along reverse trajectories—without retraining or hyperparameter tuning. Experiments demonstrate precise localization of discrete phase transition points in sampling paths, yielding substantial improvements in sampling efficiency. The method achieves state-of-the-art performance across diverse tasks, including class-conditional generation, rare-class synthesis, zero-shot image classification, and style transfer.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We analyse how the sampling dynamics of distributions evolve in score-based diffusion models using cross-fluctuations, a centered-moment statistic from statistical physics. Specifically, we show that starting from an unbiased isotropic normal distribution, samples undergo sharp, discrete transitions, eventually forming distinct events of a desired distribution while progressively revealing finer structure. As this process is reversible, these transitions also occur in reverse, where intermediate states progressively merge, tracing a path back to the initial distribution. We demonstrate that these transitions can be detected as discontinuities in $n^{ ext{th}}$-order cross-fluctuations. For variance-preserving SDEs, we derive a closed-form for these cross-fluctuations that is efficiently computable for the reverse trajectory. We find that detecting these transitions directly boosts sampling efficiency, accelerates class-conditional and rare-class generation, and improves two zero-shot tasks--image classification and style transfer--without expensive grid search or retraining. We also show that this viewpoint unifies classical coupling and mixing from finite Markov chains with continuous dynamics while extending to stochastic SDEs and non Markovian samplers. Our framework therefore bridges discrete Markov chain theory, phase analysis, and modern generative modeling.

Problem

• Analyzing sampling dynamics in diffusion models using cross-fluctuations
• Detecting phase transitions during sample generation and reversal
• Improving sampling efficiency and zero-shot task performance

Innovation

• Using cross-fluctuations to detect phase transitions
• Deriving closed-form cross-fluctuations for reverse trajectories
• Detecting transitions boosts sampling efficiency and generation