Post-Hoc Guidance for Consistency Models by Joint Flow Distribution Learning

2026-04-10

AI Summary

Consistency Models (CMs) lack a posterior-guidance mechanism that operates without a diffusion teacher, making it challenging to flexibly balance generation fidelity and diversity. This work proposes Joint Flow Distribution Learning (JFDL), a lightweight, teacher-free approach that aligns the Gaussianity of implicit noise in the unconditional and conditional velocity fields of a pretrained CM. By doing so, JFDL enables tunable guidance for consistency models that originally support only conditional generation. Experiments on CIFAR-10 and ImageNet 64×64 demonstrate that the method significantly reduces FID scores, achieving generation quality comparable to Classifier-Free Guidance (CFG), thereby unlocking flexible and controllable synthesis capabilities in CMs.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Classifier-free Guidance (CFG) lets practitioners trade-off fidelity against diversity in Diffusion Models (DMs). The practicality of CFG is however hindered by DMs sampling cost. On the other hand, Consistency Models (CMs) generate images in one or a few steps, but existing guidance methods require knowledge distillation from a separate DM teacher, limiting CFG to Consistency Distillation (CD) methods. We propose Joint Flow Distribution Learning (JFDL), a lightweight alignment method enabling guidance in a pre-trained CM. With a pre-trained CM as an ordinary differential equation (ODE) solver, we verify with normality tests that the variance-exploding noise implied by the velocity fields from unconditional and conditional distributions is Gaussian. In practice, JFDL equips CMs with the familiar adjustable guidance knob, yielding guided images with similar characteristics to CFG. Applied to an original Consistency Trained (CT) CM that could only do conditional sampling, JFDL unlocks guided generation and reduces FID on both CIFAR-10 and ImageNet 64x64 datasets. This is the first time that CMs are able to receive effective guidance post-hoc without a DM teacher, thus, bridging a key gap in current methods for CMs.

Problem

• Consistency Models
• Classifier-free Guidance
• Post-Hoc Guidance
• Diffusion Models
• Guided Generation

Innovation

• Consistency Models
• Classifier-free Guidance
• Joint Flow Distribution Learning
• Post-hoc Guidance
• Diffusion Models