Not All Prediction Targets Keep Training-Free Diffusion Guidance on the Manifold

2026-07-01

AI Summary

This work addresses a critical limitation of Training-Free Guidance (TFG) in high-noise regimes, where estimating a clean image from pure noise often causes the guidance signal to deviate from the data manifold, degrading generation quality. The study is the first to reveal that the choice of prediction target—ε, v, or x—profoundly influences TFG’s ability to preserve the data manifold. Theoretical analysis demonstrates that x-prediction directly yields the clean image, substantially reducing estimation error under high noise. To detect such manifold distortions overlooked by conventional metrics, the authors introduce a novel evaluation measure, guided-class FID (Child FID). Experiments on a newly curated fine-grained bird benchmark and style transfer tasks confirm that TFG with x-prediction significantly outperforms other prediction strategies in maintaining sample manifold consistency.

Abstract

Training-free guidance (TFG) steers a pretrained diffusion model toward a desired attribute at inference. To be effective, this guidance must be applied from the earliest, high-noise steps of sampling. Because its objective (a classifier or energy) is defined on clean images, $ε$- and $v$-prediction models must first estimate the clean image $\hat{x}$ from the noisy state at each step, and the accuracy of that estimate determines how easily guidance drifts off the data manifold. $x$-prediction, a recent alternative, outputs the clean image directly, removing this source of error even at high noise. This is our motivation. We provide a theoretical analysis of how each prediction target shapes this accuracy, and introduce guided-class FID (Child FID), a metric that exposes the manifold damage standard evaluation misses. Experiments on a new fine-grained bird benchmark and on style transfer confirm that $x$-prediction keeps guided samples on the manifold most reliably, making it the strongest foundation for training-free guidance. Code is available at https://github.com/ManLuML/on-manifold-tfg

Problem

• training-free guidance
• diffusion models
• data manifold
• prediction targets
• manifold drift

Innovation

• training-free guidance
• x-prediction
• data manifold
• guided-class FID
• diffusion models