Realtime-VLA FLASH: Speculative Inference Framework for Diffusion-based VLAs

2026-05-13

AI Summary

Diffusion-based vision-language-action (VLA) models face significant challenges in real-time deployment due to high inference latency. This work introduces speculative inference to diffusion-based VLA for the first time, proposing a framework that pairs a lightweight draft model with a primary action expert model operating in parallel, complemented by a phase-aware fallback mechanism. This approach substantially reduces latency while preserving task reliability. Evaluated on the LIBERO benchmark, the method decreases average inference latency from 58.0 ms to 19.1 ms—a 3.04× speedup—without compromising task success rates. Its practical efficacy is further demonstrated in a real-world conveyor belt sorting task.

Abstract

Diffusion-based vision-language-action models (dVLAs) are promising for embodied intelligence but are fundamentally limited in real-time deployment by the high latency of full inference. We propose Realtime-VLA FLASH, a speculative inference framework that eliminates most full inference calls during replanning by introducing a lightweight draft model with parallel verification via the main model's Action Expert and a phase-aware fallback mechanism that reverts to the full inference pipeline when needed. This design enables low-latency, high-frequency replanning without sacrificing reliability. Experiments show that on LIBERO, FLASH largely preserves task performance by replacing many 58.0 ms full-inference rounds with speculative rounds as fast as 7.8 ms, lowering task-level average inference latency to 19.1 ms (3.04x speedup). We additionally demonstrate effectiveness on real-world conveyor-belt sorting, highlighting its practical impact for latency-critical embodied tasks.

Problem

• diffusion-based VLA
• real-time deployment
• high latency
• embodied intelligence
• inference speed

Innovation

• speculative inference
• diffusion-based VLA
• real-time replanning
• low-latency control
• embodied intelligence