Problem
Research questions and friction points this paper is trying to address.
diffusion language models
granularity
planning
noise level
coarse-to-fine generation
When to Plan, When to Polish: Noise Level as a Granularity Axis for Diffusion Language Models
🤖 AI Summary
This work addresses the limitation of standard autoregressive diffusion language models in constructing an early semantic scaffold during high-noise stages, which hinders planning-sensitive text generation. The authors propose Noise-Dependent Granularity Control (NDGC), a novel approach that directly leverages the noise level as a signal to dynamically modulate generation granularity throughout the denoising process: at high noise levels, coherent multi-word phrases are generated to support coarse-grained semantic planning, while at low noise levels, the model reverts to token-level refinement. Notably, NDGC achieves this coarse-to-fine, planner-like generation without requiring an explicit planner or hierarchical architecture. Experimental results demonstrate that NDGC significantly improves the timeliness of semantic scaffold construction, the coherence of content restoration, and overall generation quality on tasks such as WritingPrompts.
📝 Abstract
Standard tokenwise diffusion LMs keep training corruption and inference commitment at token granularity throughout denoising. At high noise, this leaves scattered local fragments rather than coherent evidence, making it hard to form early coarse structure, exactly what planning-sensitive generation requires. Hierarchical planning methods add coarse stages to separate planning from wording, but they need extra planners, block latents, or two stage designs. We propose Noise Dependent Granularity Control (NDGC), a single-level diffusion method that uses the noise level as a granularity cue. NDGC aligns training exposure and inference commitment with denoising progress. High noise steps use coherent token groups to support early meaning commitment, while low noise steps return to token level refinement. This creates planning like coarse to fine denoising without an explicit planner or hierarchical architecture. Across controlled tests, ablations, and WritingPrompts, NDGC shows earlier skeleton formation, better ordered recovery, and healthier outputs.
Innovation
Methods, ideas, or system contributions that make the work stand out.
diffusion language models
granularity control
noise level
coarse-to-fine generation
planning-aware generation
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
