DreamReasoner-8B: Block-Size Curriculum Learning for Diffusion Reasoning Models

2026-06-17

AI Summary

This work addresses the significant performance degradation of chunked diffusion language models in long-chain reasoning tasks, which arises from a mismatch between the fixed training chunk size and the variable demands of inference. The study is the first to reveal the critical impact of training chunk size on reasoning capability and proposes a chunk-size curriculum learning strategy that progressively trains the model from fine-grained to coarse-grained chunks, effectively bridging the granularity gap between training and inference. By integrating parallel chunk denoising with chain-of-thought reasoning, the proposed approach enables the DreamReasoner-8B model to achieve mathematical and code reasoning performance on par with state-of-the-art autoregressive models such as Qwen3-8B, thereby demonstrating its efficacy.

Abstract

Block diffusion language models accelerate decoding through parallel block-wise denoising, yet whether they can be reliably scaled for long chain-of-thought (CoT) reasoning remains unresolved. To this end, we develop DreamReasoner-8B, an open-source block diffusion reasoning model, and conduct a systematic study of how training and inference block sizes affect long-CoT reasoning. Our analysis reveals a stark performance disparity: training with large block sizes yields remarkably poor reasoning, whereas small block sizes preserve effective reasoning. To bridge this granularity gap, we propose block-size curriculum learning, which gradually transitions training from fine-grained to coarse-grained block sizes, thereby overcoming this limitation and enabling strong reasoning performance that generalizes across diverse inference block sizes. On mathematical and code reasoning benchmarks, DreamReasoner-8B achieves results competitive with leading open autoregressive models such as Qwen3-8B. This work establishes a practical foundation for efficient, reasoning-capable diffusion language models. We release our model at https://github.com/DreamLM/DreamReasoner.

Problem

• block diffusion language models
• chain-of-thought reasoning
• block size
• reasoning scalability
• diffusion models

Innovation

• block diffusion language models
• chain-of-thought reasoning
• block-size curriculum learning
• parallel decoding
• reasoning-capable diffusion models