Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesis

2025-07-24

AI Summary

Existing distribution matching distillation (DMD) methods rely on reverse KL divergence minimization, which suffers from mode collapse and limits compression efficacy for both one-step and multi-step diffusion models. To address this, we propose an adversarial distribution matching distillation framework: a diffusion-based discriminator is designed to jointly align score estimates of real and generated data in both latent and pixel spaces; KL optimization is replaced by an adversarial mechanism to ensure full support coverage of the target distribution; and an ODE-sampled distribution loss is integrated with multi-space discrimination for joint optimization. On SDXL, our one-step distilled model achieves superior FID over DMD2 with significant inference speedup. For multi-step distillation, our method attains state-of-the-art performance on SD3-Medium, SD3.5-Large, and CogVideoX—demonstrating its generality and superiority for efficient image and video synthesis.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Distributed Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Distribution Matching Distillation (DMD) is a promising score distillation technique that compresses pre-trained teacher diffusion models into efficient one-step or multi-step student generators. Nevertheless, its reliance on the reverse Kullback-Leibler (KL) divergence minimization potentially induces mode collapse (or mode-seeking) in certain applications. To circumvent this inherent drawback, we propose Adversarial Distribution Matching (ADM), a novel framework that leverages diffusion-based discriminators to align the latent predictions between real and fake score estimators for score distillation in an adversarial manner. In the context of extremely challenging one-step distillation, we further improve the pre-trained generator by adversarial distillation with hybrid discriminators in both latent and pixel spaces. Different from the mean squared error used in DMD2 pre-training, our method incorporates the distributional loss on ODE pairs collected from the teacher model, and thus providing a better initialization for score distillation fine-tuning in the next stage. By combining the adversarial distillation pre-training with ADM fine-tuning into a unified pipeline termed DMDX, our proposed method achieves superior one-step performance on SDXL compared to DMD2 while consuming less GPU time. Additional experiments that apply multi-step ADM distillation on SD3-Medium, SD3.5-Large, and CogVideoX set a new benchmark towards efficient image and video synthesis.

Problem

• Address mode collapse in diffusion distillation via adversarial alignment
• Improve one-step distillation with hybrid latent-pixel discriminators
• Enhance pre-training using distributional loss for better initialization

Innovation

• Adversarial Distribution Matching for score distillation
• Hybrid discriminators in latent and pixel spaces
• Distributional loss on ODE pairs for initialization