Discrete Beckmann Transport Models for One-Step Language Modeling and Reasoning

2026-09-14

AI Summary

该研究通过引入离散贝克曼传输模型(DBTM)，解决了多步采样压缩问题，实现了一步生成高质量文本，无需预训练教师模型。

Abstract

Discrete diffusion and flow models are a promising alternative to autoregressive language models, but compressing many-step sampling into fewer steps typically requires distilling a pretrained teacher model. This caps the student at the teacher's quality and requires a costly two-stage training pipeline. We introduce Discrete Beckmann Transport Models (DBTM), built on a time-independent flow whose autonomous transport map provably carries any point in the ambient space to a fixed point on the vertices of the simplex in a single step. We show that this fixed-point property is characterized by a conservation equation whose residual can be minimized directly from data, removing the requirement for a teacher flow and time conditioning. Under this construction, a partially trained map corresponds to the flow truncated at finite time, so generation reduces to iterating one map until it reaches a fixed point. We further extend the map to a partial-context interpolant where additional function evaluations act as refinement steps rather than ODE integration steps. On language modeling and reasoning tasks, DBTM enables one- and few-step generation that improves quality and accuracy over discrete diffusion and continuous flow baselines.

Problem

• Discrete Diffusion
• Flow Models
• Autoregressive Language Models
• Knowledge Distillation

Innovation

• Discrete Beckmann Transport Models (DBTM)
• fixed-point property
• conservation equation
• one-step generation
• partial-context interpolant