Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding

2026-07-06

AI Summary

Traditional language models struggle to balance generation quality and inference efficiency across diverse deployment scenarios. This work proposes a unified tri-modal language model that, for the first time, integrates autoregressive (AR), diffusion, and speculative decoding within a single architecture. By jointly training AR and diffusion objectives, the model reveals their complementary nature. In speculative mode, it leverages diffusion to generate draft tokens and AR for verification, substantially improving acceptance rates and hardware efficiency. Evaluated on multi-scale variants (3B/8B/14B) with vision-language extensions, Nemotron-Labs-Diffusion-8B matches Qwen3-8B in accuracy while generating six times more tokens per forward pass and achieving a 4× throughput gain on GB200 GPUs, outperforming all existing open-source AR and diffusion models.

Abstract

We introduce Nemotron-Labs-Diffusion, a tri-mode language model (LM) that unifies AR, diffusion, and self-speculation decoding within a single architecture. Trained with a joint AR-diffusion objective, Nemotron-Labs-Diffusion can switch modes to sustain high throughput across deployment settings and concurrency levels. Our study shows that (1) AR and diffusion objectives are complementary: diffusion improves lookahead planning, while AR provides left-to-right linguistic priors. (2) In self-speculation mode, diffusion drafts while AR verifies, outperforming multi-token prediction (MTP) methods in both acceptance rate and real-device efficiency. (3) A speed-of-light analysis further demonstrates diffusion's long-term potential, with up to 76.5% more tokens per forward pass than self-speculation under an optimal sampler. Scaling to 3B, 8B, and 14B parameters, our Nemotron-Labs-Diffusion family, including base, instruct, and vision-language models, consistently outperforms state-of-the-art open-source AR and diffusion LMs in both accuracy and speed. For example, Nemotron-Labs-Diffusion-8B decodes 6x more tokens per forward than Qwen3-8B with comparable accuracy, translating to 4x higher throughput on SPEED-Bench with SGLang on a GB200 GPU.

Problem

• language model
• autoregressive
• diffusion
• self-speculation
• decoding

Innovation

• tri-mode language model
• diffusion decoding
• self-speculation
• autoregressive-diffusion unification
• high-throughput inference