One-step Diffusion Models with Bregman Density Ratio Matching

2025-10-19

AI Summary

Diffusion and flow models suffer from low inference efficiency and high computational cost due to multi-step sampling. To address this, we propose a single-step diffusion generative framework grounded in Bregman divergence-based density ratio matching. Our method unifies diffusion distillation as a density ratio matching problem under Bregman divergence, offering a convex-analytic interpretation that harmonizes diverse existing distillation objectives and establishes a theoretically consistent foundation for acceleration. By integrating characteristics of both diffusion and flow models, we design a compact training objective for the student generator. On CIFAR-10 and text-to-image generation tasks, our single-step sampling achieves significantly better FID scores than reverse-KL distillation while preserving visual fidelity comparable to the teacher model. This work jointly advances inference efficiency and theoretical rigor in diffusion model acceleration.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Web Mining and Content Analysis: Content-based information diffusion
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Diffusion and flow models achieve high generative quality but remain computationally expensive due to slow multi-step sampling. Distillation methods accelerate them by training fast student generators, yet most existing objectives lack a unified theoretical foundation. In this work, we propose Di-Bregman, a compact framework that formulates diffusion distillation as Bregman divergence-based density-ratio matching. This convex-analytic view connects several existing objectives through a common lens. Experiments on CIFAR-10 and text-to-image generation demonstrate that Di-Bregman achieves improved one-step FID over reverse-KL distillation and maintains high visual fidelity compared to the teacher model. Our results highlight Bregman density-ratio matching as a practical and theoretically-grounded route toward efficient one-step diffusion generation.

Problem

• Accelerating slow multi-step diffusion sampling
• Providing unified theoretical foundation for distillation
• Achieving efficient one-step generation with fidelity

Innovation

• One-step diffusion distillation via Bregman divergence
• Density-ratio matching for unified theoretical framework
• Efficient one-step generation with improved visual fidelity