Devil is in the Details: Density Guidance for Detail-Aware Generation with Flow Models

2025-02-09

AI Summary

In diffusion models, a fundamental trade-off exists between image likelihood and perceptual quality: high-likelihood samples tend to be oversmoothed and lack fine details, whereas low-likelihood samples exhibit rich detail but suffer from structural distortions. To address this, we propose a density-guidance mechanism that enables provably exact and continuous control of log-density during sampling within the continuous normalizing flow (CNF) framework. We introduce the *score alignment condition*, the first formal criterion explaining the efficacy of prior-guided sampling. Our method is compatible with both deterministic (ODE) and stochastic (SDE) samplers and supports fine-grained, interpretable balancing of realism and detail fidelity. Evaluated on multiple benchmark datasets, our approach significantly enhances texture sharpness and structural clarity while preserving high sample quality, achieving near-zero log-density control error.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Diffusion models have emerged as a powerful class of generative models, capable of producing high-quality images by mapping noise to a data distribution. However, recent findings suggest that image likelihood does not align with perceptual quality: high-likelihood samples tend to be smooth, while lower-likelihood ones are more detailed. Controlling sample density is thus crucial for balancing realism and detail. In this paper, we analyze an existing technique, Prior Guidance, which scales the latent code to influence image detail. We introduce score alignment, a condition that explains why this method works and show that it can be tractably checked for any continuous normalizing flow model. We then propose Density Guidance, a principled modification of the generative ODE that enables exact log-density control during sampling. Finally, we extend Density Guidance to stochastic sampling, ensuring precise log-density control while allowing controlled variation in structure or fine details. Our experiments demonstrate that these techniques provide fine-grained control over image detail without compromising sample quality.

Problem

• Control image detail density
• Enhance realism and detail balance
• Ensure precise log-density control

Innovation

• Density Guidance for detail control
• Score alignment in flow models
• Exact log-density during sampling