Recursive Scaling in Masked Diffusion Models

2026-06-16

AI Summary

Traditional Masked Diffusion Models (MDMs) scale performance by increasing model parameters or denoising steps, resulting in poor computational and parameter efficiency. This work proposes the Recursive Masked Diffusion Model (R-MDM), which recursively reuses the same denoising Transformer within each diffusion step, treating recursion depth as a third axis of model scaling. This approach effectively increases model depth without expanding parameter count and reduces the number of forward passes during inference. On structured generation tasks such as Sudoku and Countdown, R-MDM achieves performance comparable to non-recursive baselines with approximately L times more parameters using only L recursive steps, substantially improving both parameter and inference efficiency.

Abstract

Masked diffusion models (MDMs) have recently emerged as a promising paradigm for sequence generation. Scaling MDMs is conventionally achieved by increasing the parameter count or the number of denoising steps. We introduce Recursive Masked Diffusion Models (R-MDMs), which add recursive depth as a third scaling axis by repeatedly applying the same denoising transformer within each diffusion step. Recursion enables iterative refinement of the output through parameter reuse, increasing effective model depth without increasing parameter count. Across structured generation tasks, including Sudoku and Countdown, we show that R-MDMs achieve substantially improved parameter efficiency: a model with $L$ recursive iterations often matches the performance of non-recursive baselines with roughly $L\times$ more parameters. Moreover, recursive refinement can partially substitute for additional denoising steps, allowing recursive models to reach the same generation quality with fewer forward passes at inference time. These results suggest that recursive depth is a practically useful scaling mechanism for MDMs, improving both parameter efficiency and the allocation of test-time compute.

Problem

• Masked Diffusion Models
• Recursive Scaling
• Parameter Efficiency
• Sequence Generation
• Model Scaling

Innovation

• Recursive Masked Diffusion
• Parameter Efficiency
• Iterative Refinement
• Scaling Law
• Diffusion Models