EVODiff: Entropy-aware Variance Optimized Diffusion Inference

2025-09-30

AI Summary

Diffusion models suffer from slow inference and training-inference inconsistency. To address these bottlenecks, this paper proposes an information-theoretic framework for efficient generation. First, it reveals the inherent conditional entropy reduction during denoising, theoretically justifying data prediction over noise prediction. Second, it introduces a reference-free conditional variance optimization mechanism that jointly minimizes conditional entropy and variance in the reverse process, systematically reducing generation uncertainty. The method integrates information-theoretic analysis, data-prediction parameterization, and adaptive variance control. Experiments demonstrate significant improvements: on CIFAR-10, FID drops from 5.10 to 2.78 using only 10 sampling steps, with a 45.5% reduction in reconstruction error; on ImageNet-256, generation quality is preserved while reducing function evaluations by 25%; in text-to-image synthesis, the approach substantially suppresses artifacts.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Diffusion models (DMs) excel in image generation, but suffer from slow inference and the training-inference discrepancies. Although gradient-based solvers like DPM-Solver accelerate the denoising inference, they lack theoretical foundations in information transmission efficiency. In this work, we introduce an information-theoretic perspective on the inference processes of DMs, revealing that successful denoising fundamentally reduces conditional entropy in reverse transitions. This principle leads to our key insights into the inference processes: (1) data prediction parameterization outperforms its noise counterpart, and (2) optimizing conditional variance offers a reference-free way to minimize both transition and reconstruction errors. Based on these insights, we propose an entropy-aware variance optimized method for the generative process of DMs, called EVODiff, which systematically reduces uncertainty by optimizing conditional entropy during denoising. Extensive experiments on DMs validate our insights and demonstrate that our method significantly and consistently outperforms state-of-the-art (SOTA) gradient-based solvers. For example, compared to the DPM-Solver++, EVODiff reduces the reconstruction error by up to 45.5% (FID improves from 5.10 to 2.78) at 10 function evaluations (NFE) on CIFAR-10, cuts the NFE cost by 25% (from 20 to 15 NFE) for high-quality samples on ImageNet-256, and improves text-to-image generation while reducing artifacts. Code is available at https://github.com/ShiguiLi/EVODiff.

Problem

• Optimizes diffusion model inference to reduce conditional entropy
• Addresses slow inference and training-inference discrepancies in DMs
• Minimizes transition and reconstruction errors through variance optimization

Innovation

• Optimizes conditional entropy in diffusion model denoising
• Uses data prediction parameterization for better performance
• Minimizes transition and reconstruction errors by variance optimization