CFG-Zero*: Improved Classifier-Free Guidance for Flow Matching Models

2025-03-24

AI Summary

In flow matching models, classifier-free guidance (CFG) often misguides sampling trajectories during early training stages due to inaccurate velocity estimation, degrading both image fidelity and controllability. To address this, we propose CFG-Zero*, the first method introducing a learnable scaling factor to correct velocity estimation bias, coupled with a zero-initialization strategy that skips the initial steps of the ODE solver to avoid early misdirection. CFG-Zero* is fully compatible with mainstream flow matching architectures—including Lumina-Next, SD3, Flux, and Wan-2.1—without requiring any modifications to the training pipeline. Extensive experiments on text-to-image and text-to-video generation demonstrate that CFG-Zero* consistently outperforms standard CFG across multiple benchmarks, yielding substantial improvements in visual quality and conditional adherence. The implementation is publicly available.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Classifier-Free Guidance (CFG) is a widely adopted technique in diffusion/flow models to improve image fidelity and controllability. In this work, we first analytically study the effect of CFG on flow matching models trained on Gaussian mixtures where the ground-truth flow can be derived. We observe that in the early stages of training, when the flow estimation is inaccurate, CFG directs samples toward incorrect trajectories. Building on this observation, we propose CFG-Zero*, an improved CFG with two contributions: (a) optimized scale, where a scalar is optimized to correct for the inaccuracies in the estimated velocity, hence the * in the name; and (b) zero-init, which involves zeroing out the first few steps of the ODE solver. Experiments on both text-to-image (Lumina-Next, Stable Diffusion 3, and Flux) and text-to-video (Wan-2.1) generation demonstrate that CFG-Zero* consistently outperforms CFG, highlighting its effectiveness in guiding Flow Matching models. (Code is available at github.com/WeichenFan/CFG-Zero-star)

Problem

• Improving CFG accuracy in flow matching models
• Correcting early training trajectory errors in CFG
• Enhancing image and video generation with CFG-Zero*

Innovation

• Optimized scale corrects velocity inaccuracies
• Zero-init for first ODE solver steps
• Improved CFG for flow matching models