Discrete Diffusion Inference-Time Control with Nested Sequential Monte Carlo

2026-08-20

AI Summary

本文研究了离散扩散语言模型中的推理时间控制问题，通过嵌套序列蒙特卡洛方法来指导文本生成以获得序列级奖励。

Abstract

We study inference-time control for text generation in discrete diffusion language models, where the goal is to steer sampling toward sequence-level rewards without retraining. Prior work in this domain has focused on particle-based methods such as best-of-$n$ sampling and bootstrap sequential Monte Carlo, which may suffer from overoptimism and weight degeneracy, respectively. We address these limitations using \emph{nested} sequential Monte Carlo methods. We formulate nested SMC (NSMC) and fully-adapted nested SMC (FA-NSMC) for Feynman--Kac steering, identifying and correcting errors in prior formulations that lead to biased final estimates. We evaluate these methods on toxicity and fluency steering tasks, showing that NSMC and FA-NSMC consistently outperform best-of-$n$ and bootstrap SMC.

Problem

• discrete diffusion
• inference-time control
• sequence-level rewards
• text generation

Innovation

• nested sequential Monte Carlo
• discrete diffusion language models
• inference-time control
• Feynman--Kac steering