BlockVLA: Accelerating Autoregressive VLA via Block Diffusion Finetuning

2026-05-13

AI Summary

This work addresses the high latency and error accumulation inherent in autoregressive vision-language-action (VLA) models for long-horizon robotic tasks, as well as the limitations of existing discrete diffusion approaches that suffer from repeated denoising and incompatibility with key-value (KV) caching. The authors propose BlockVLA, a framework that reformulates pretrained autoregressive models into efficient policies through a block-wise diffusion paradigm. This approach preserves causal dependencies across blocks while enabling parallel denoising within each block and supports prefix KV cache reuse to reduce iterative computational overhead. Seamlessly bridging autoregressive pretraining and diffusion fine-tuning, BlockVLA achieves 3.3× faster inference than standard diffusion baselines on LIBERO and SimplerEnv benchmarks, converges more rapidly during training, and substantially improves early success rates on complex long-horizon tasks.

Abstract

While autoregressive (AR) Vision-Language-Action (VLA) models have demonstrated formidable reasoning capabilities in robotic tasks, their sequential decoding process often incurs high inference latency and may amplify error accumulation during long-horizon execution. Discrete Diffusion Language Models (dLLMs) provide a promising alternative through parallel token refinement, but their practical deployment in robotics remains limited by repeated denoising function evaluations (NFEs) and the difficulty of directly applying standard KV caching to bidirectional iterative decoding. To bridge these paradigms, we propose BlockVLA, a framework that adapts pretrained AR backbones into an efficient discrete diffusion policy through a block diffusion paradigm. BlockVLA maintains autoregressive dependencies at the block level while enabling parallel denoising within each block, thereby combining global causal coherence with local parallel generation. This design enables prefix KV-cache reuse across completed blocks, reduces the effective cost of iterative denoising, and provides a smoother transition from AR pretraining to diffusion-based policy fine-tuning. We conduct extensive evaluations on the LIBERO and SimplerEnv benchmarks. Experimental results demonstrate that our BlockVLA achieves a 3.3$\times$ inference acceleration over standard discrete diffusion baselines. Furthermore, our model exhibits superior training efficiency, with success rates converging substantially faster than baselines, a gain that is particularly pronounced in complex, long-horizon tasks, where BlockVLA achieves significant performance gains in the early stages of training. This work establishes Block Diffusion as a robust bridge between large-scale pretrained AR models and efficient, high-frequency real-time robotic control.

Problem

• autoregressive VLA
• inference latency
• error accumulation
• discrete diffusion
• KV caching

Innovation

• Block Diffusion
• Autoregressive VLA
• Parallel Denoising
• KV Caching
• Robotic Policy Fine-tuning