PairFlow: Closed-Form Source-Target Coupling for Few-Step Generation in Discrete Flow Models

2025-12-23

AI Summary

Discrete flow models (DFMs) suffer from slow sampling due to iterative procedures, and existing acceleration methods typically rely on costly fine-tuning. This paper proposes the first lightweight pre-processing framework for DFMs that requires no pre-trained teacher model. Our core innovations are: (1) the first closed-form invertible transformation designed specifically for DFMs, enabling explicit coupling between source and target distributions during training; (2) achieving few-step sampling with only 1.7% additional computational overhead; (3) significantly accelerating sampling—on molecular structures, binary images, and RGB images—while preserving or even surpassing baseline generation quality; and (4) learning representations more amenable to subsequent knowledge distillation, demonstrating strong cross-domain generalization. By eliminating the need for two-stage fine-tuning, our method achieves high efficiency, broad applicability across discrete data domains, and practical deployability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We introduce $ exttt{PairFlow}$, a lightweight preprocessing step for training Discrete Flow Models (DFMs) to achieve few-step sampling without requiring a pretrained teacher. DFMs have recently emerged as a new class of generative models for discrete data, offering strong performance. However, they suffer from slow sampling due to their iterative nature. Existing acceleration methods largely depend on finetuning, which introduces substantial additional training overhead. $ exttt{PairFlow}$ addresses this issue with a lightweight preprocessing step. Inspired by ReFlow and its extension to DFMs, we train DFMs from coupled samples of source and target distributions, without requiring any pretrained teacher. At the core of our approach is a closed-form inversion for DFMs, which allows efficient construction of paired source-target samples. Despite its extremely low cost, taking only up to 1.7% of the compute needed for full model training, $ exttt{PairFlow}$ matches or even surpasses the performance of two-stage training involving finetuning. Furthermore, models trained with our framework provide stronger base models for subsequent distillation, yielding further acceleration after finetuning. Experiments on molecular data as well as binary and RGB images demonstrate the broad applicability and effectiveness of our approach.

Problem

• Accelerates sampling in discrete flow models
• Reduces training overhead without pretrained teacher
• Enables efficient source-target coupling via closed-form inversion

Innovation

• Lightweight preprocessing step for Discrete Flow Models
• Closed-form inversion for efficient source-target coupling
• Achieves few-step sampling without pretrained teacher