Posterior Refinement: Fast Language Generation via Any-Order Flow Maps

2026-06-23

AI Summary

Existing non-autoregressive language generation models struggle to balance speed and quality in parallel multi-token generation, often suffering from factorization errors or rigid inference dynamics. This work proposes the FMLM+ framework, which introduces posterior consistency scoring into Flow-based Mapping Language Models (FMLM) for the first time. By integrating masked noise scheduling with arbitrary-order joint sequence transport, FMLM+ enables single-step full-sequence generation while supporting adaptive posterior refinement. The method drastically reduces the number of function evaluations—requiring only 1/32 of those needed by baseline approaches—and achieves superior performance over MDM and prior FMLM variants across multiple benchmarks, matching or even surpassing the generation quality of discrete autoregressive baselines.

Abstract

Non-autoregressive generation offers a powerful paradigm for iterative refinement, allowing models to recursively critique, erase and regenerate arbitrary subsets of tokens. However, existing non-autoregressive models fail to realize this potential. Masked Diffusion Models (MDMs) suffer from factorization error, causing sample quality to collapse when generating multiple tokens simultaneously. Flow Map Language Models (FMLMs) circumvent this bottleneck via joint sequence transport for excellent few-step generation, but sacrifice the inference-time flexibility of MDMs. We introduce FMLM+, a framework that bridges this gap by equipping FMLM with masking-style noise schedules. While generating the full sequence in a single step, FMLM+ simultaneously scores the global consistency of each token a posteriori. We leverage this to introduce Posterior Refinement, a novel inference-time refinement strategy that enables the model to adaptively self-correct its outputs, matching the performance of discrete baselines with 32x fewer NFEs. Across diverse benchmarks, we demonstrate that FMLM+ with Posterior Refinement improves the speed--quality tradeoff over both MDM and FMLM families, providing a scalable foundation for high-fidelity language modeling.

Problem

• non-autoregressive generation
• factorization error
• inference-time flexibility
• masked diffusion models
• flow map language models

Innovation

• Posterior Refinement
• Flow Map Language Models
• Non-autoregressive Generation
• Masked Diffusion Models
• Inference-time Self-correction