Seeing the End at Step Zero: Accelerating Diffusion MLLMs via MLP Sparsity-Aware Truncation

2026-07-16

AI Summary

Diffusion-based multimodal large language models suffer from substantial redundant computation due to fixed-length generation, severely limiting inference efficiency. This work makes the first observation that a sharp sparsity shift in MLP activations during the initial denoising step strongly correlates with the boundary of semantically meaningful content. Building on this insight, the authors propose Seer, a training-free, single-step truncation framework that leverages signal-to-noise ratio–guided boundary detection and dynamic sequence truncation to terminate redundant suffix computation early. Seer further incorporates a hybrid execution strategy to enable dynamic batching. Evaluated across nine benchmarks, Seer matches or improves performance—e.g., increasing DocVQA score from 63.52 to 63.66—while achieving up to a 31× throughput gain.

Abstract

Diffusion Multimodal Large Language Models (DMLLMs) are highly effective for multimodal reasoning, yet their inference efficiency is significantly hindered by fixed-length generation constraints. Since the actual output length is unknown, output sequences are padded to a predefined maximum length, resulting in substantial redundant computation over unnecessary [EOS] tokens. In this work, we discover that DMLLMs implicitly reveal their valid semantic boundary at the very first denoising step through a distinct shift in MLP activation sparsity. Leveraging this observation, we propose Seer, a training-free framework that detects this boundary using a Signal-to-Noise Ratio (SNR)-based criterion and performs one-shot truncation of the redundant suffix for all subsequent computations. To preserve these theoretical gains during batched serving, Seer incorporates a hybrid execution strategy that maximizes throughput while seamlessly accommodating dynamic sequence lengths. Experimental results demonstrate that Seer effectively eliminates padding waste, accelerating throughput by up to $\sim$31$\times$. Across 9 benchmarks, Seer robustly maintains overall performance and even improves accuracy on complex visual tasks by mitigating noise leakage (e.g., DocVQA score increases from 63.52 to 63.66), offering a highly efficient, plug-and-play solution for DMLLM acceleration.

Problem

• Diffusion Multimodal Large Language Models
• inference efficiency
• fixed-length generation
• redundant computation
• output padding

Innovation

• MLP sparsity
• early truncation
• diffusion MLLMs
• training-free acceleration
• dynamic sequence length