TCFG: Tangential Damping Classifier-free Guidance

2025-03-23

AI Summary

In classifier-free guidance (CFG), unconditional score estimates perturb conditional manifold trajectories, causing distortion and conditional misalignment in text-to-image generation. To address this, we propose Tangential Damping Guidance (TDG): a geometric framework that models the unconditional score as the tangential component of the conditional manifold. TDG jointly filters conditional and unconditional scores via singular value decomposition (SVD), enabling manifold-aligned, nonlinear guidance. Unlike conventional CFG—which relies on linear interpolation—TDG operates intrinsically on the data manifold, preserving trajectory coherence and semantic fidelity. Our method achieves substantial improvements in image quality, text–image alignment, and manifold adherence of sampling trajectories, with virtually no additional computational overhead. The implementation is publicly available.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning with Manifolds
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Diffusion models have achieved remarkable success in text-to-image synthesis, largely attributed to the use of classifier-free guidance (CFG), which enables high-quality, condition-aligned image generation. CFG combines the conditional score (e.g., text-conditioned) with the unconditional score to control the output. However, the unconditional score is in charge of estimating the transition between manifolds of adjacent timesteps from $x_t$ to $x_{t-1}$, which may inadvertently interfere with the trajectory toward the specific condition. In this work, we introduce a novel approach that leverages a geometric perspective on the unconditional score to enhance CFG performance when conditional scores are available. Specifically, we propose a method that filters the singular vectors of both conditional and unconditional scores using singular value decomposition. This filtering process aligns the unconditional score with the conditional score, thereby refining the sampling trajectory to stay closer to the manifold. Our approach improves image quality with negligible additional computation. We provide deeper insights into the score function behavior in diffusion models and present a practical technique for achieving more accurate and contextually coherent image synthesis.

Problem

• Enhance CFG performance using geometric perspective
• Align unconditional score with conditional score
• Improve image quality with minimal computation

Innovation

• Leverages geometric perspective on unconditional score
• Filters singular vectors using SVD alignment
• Refines sampling trajectory for manifold proximity