Acting While Understanding: Asynchronous Semantic-Action Decoupling for Real-Time Vision-Language-Action Models

2026-06-13

AI Summary

This work addresses the challenge of high computational overhead in existing vision-language-action (VLA) models, which hinders low-latency, high-frequency closed-loop robotic control. The authors propose an asynchronous semantic-action decoupling framework that separates semantic understanding from action generation without modifying the VLA backbone or introducing external planners. In this framework, semantic conditions are updated at a low frequency, while the action module outputs control commands at a high frequency. To mitigate performance degradation caused by semantic lag, the method leverages historical action conditioning and temporally misaligned training. Relying solely on internal VLA interfaces, this minimally invasive approach enables high-frequency control, achieving an action module inference throughput of 35.6 Hz. Experiments on the LIBERO benchmark and real-world robots demonstrate its effectiveness in significantly improving real-time control performance.

Abstract

Vision-Language-Action models (VLAs) have demonstrated strong task understanding and generalization in robotic manipulation, yet the high computational cost of full-model inference limits their deployment in low-latency, high-frequency closed-loop control. We propose an asynchronous semantic-action decoupling framework that separates semantic understanding from action generation along the internal semantic-action interface of existing VLAs, without redesigning the vision-language backbone or introducing an external planner. A low-frequency understanding module asynchronously updates reusable semantic conditions, while a high-frequency action module continuously outputs control actions without repeatedly invoking the full model. To mitigate the temporal mismatch between stale semantics and the current execution state, we further introduce historical action conditioning and time-misalignment training, which provide short-horizon execution context and improve feedback control robustness under stale semantic conditions. Experiments on LIBERO with $π_{0.5}$ and UniVLA, together with real-robot deployment using UniVLA, show that the proposed framework achieves up to 35.6 Hz server-side action-module inference throughput and offers a low-intrusion path to high-frequency closed-loop control without running full VLA inference at control rate.

Problem

• Vision-Language-Action models
• real-time control
• low-latency inference
• closed-loop control
• computational cost

Innovation

• asynchronous decoupling
• vision-language-action models
• real-time control
• semantic-action interface
• time-misalignment training