Condition Errors Refinement in Autoregressive Image Generation with Diffusion Loss

2026-02-02

AI Summary

This work addresses the issue of conditional inconsistency in autoregressive image generation, where conditioning errors often degrade output quality and stability. The authors theoretically establish, for the first time, that under a block denoising optimization framework, autoregressive conditional generation can induce exponential decay of conditioning errors. To further enhance fidelity, they propose a conditional refinement mechanism grounded in optimal transport theory, which steers the generative process toward the target conditional distribution via Wasserstein gradient flows. Experimental results demonstrate that the proposed method significantly outperforms existing diffusion models and autoregressive models augmented with diffusion-based losses, both in terms of image quality and conditional consistency.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Recent studies have explored autoregressive models for image generation, with promising results, and have combined diffusion models with autoregressive frameworks to optimize image generation via diffusion losses. In this study, we present a theoretical analysis of diffusion and autoregressive models with diffusion loss, highlighting the latter's advantages. We present a theoretical comparison of conditional diffusion and autoregressive diffusion with diffusion loss, demonstrating that patch denoising optimization in autoregressive models effectively mitigates condition errors and leads to a stable condition distribution. Our analysis also reveals that autoregressive condition generation refines the condition, causing the condition error influence to decay exponentially. In addition, we introduce a novel condition refinement approach based on Optimal Transport (OT) theory to address ``condition inconsistency''. We theoretically demonstrate that formulating condition refinement as a Wasserstein Gradient Flow ensures convergence toward the ideal condition distribution, effectively mitigating condition inconsistency. Experiments demonstrate the superiority of our method over diffusion and autoregressive models with diffusion loss methods.

Problem

• condition errors
• condition inconsistency
• autoregressive image generation
• diffusion loss

Innovation

• autoregressive diffusion
• condition error refinement
• optimal transport
• Wasserstein gradient flow
• diffusion loss