KERV: Kinematic-Rectified Speculative Decoding for Embodied VLA Models

2026-03-02

AI Summary

This work addresses the slow inference speed of current Vision-Language-Action (VLA) models and the challenges associated with speculative decoding, particularly high re-execution overhead and difficulty in tuning acceptance thresholds. To overcome these limitations, the study introduces robot kinematics into the speculative decoding process for the first time, proposing a kinematics-informed speculative decoding framework. This framework leverages kinematics-driven Kalman filtering to predict actions and compensate for token-level errors, while also incorporating a dynamic threshold adjustment strategy to enable an adaptive acceptance mechanism. Evaluated across diverse tasks and environments, the method achieves inference speedups of 27% to 37% with negligible degradation in task success rates.

Technology Category

• Computer Vision: Language and Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Vision-Language-Action (VLA) models build a token-domain robot control paradigm, yet suffer from low speed. Speculative Decoding (SD) is an optimization strategy that can boost inference speed. Two key issues emerge when integrating VLA and SD: first, SD relies on re-inference to address token errors, which is computationally expensive; second, to mitigate token errors, the acceptance threshold in SD requires careful adjustment. Existing works fail to address the above two issues effectively. Meanwhile, as the bridge between AI and the physical world, existing embodied intelligence has overlooked the application of robotic kinematics. To address these issues, we innovatively combine token-domain VLA models with kinematic-domain prediction for SD, proposing a kinematic-rectified SD framework named KERV. We employ a kinematics-based Kalman Filter to predict actions and compensate for SD errors, avoiding costly re-inference. Moreover, we design a kinematics-based adjustment strategy to dynamically rectify the acceptance threshold, addressing the difficulty of threshold determination. Experimental results across diverse tasks and environments demonstrate that KERV achieves 27%~37% acceleration with nearly no Success Rate loss.

Problem

• Speculative Decoding
• Vision-Language-Action models
• token errors
• acceptance threshold
• robotic kinematics

Innovation

• Speculative Decoding
• Kinematic Rectification
• Vision-Language-Action Models
• Kalman Filter
• Embodied Intelligence