Jetson-PI: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference

2026-07-14

AI Summary

Deploying vision-language-action (VLA) models on low-power onboard platforms such as the Jetson Orin is challenged by high latency and misalignment between perception and execution. This work proposes Jetson-PI, which introduces a lightweight future-correction module to predict action-conditioned future environment representations, enabling proactive asynchronous alignment. Furthermore, it incorporates a confidence-driven dynamic scheduling strategy combined with system-level optimizations—including CUDA graph reuse, GPU-resident buffers, and stream unfolding—to significantly enhance real-time performance. Evaluated on the Jetson Orin, the approach achieves an 8.66× and 5.41× improvement in control frequency over PyTorch and vla.cpp, respectively, and surpasses VLASH by 14.8% in average task success rate on the LIBERO benchmark.

Abstract

Vision-Language-Action (VLA) models have achieved impressive performance on diverse embodied tasks. However, deploying VLA models on low-power onboard devices, such as the Jetson Orin, remains challenging due to their high computational complexity, which leads to substantial inference latency and low control frequency. Asynchronous inference can partially mask this latency by parallelizing action execution and subsequent inference, but it introduces two critical issues: perception-execution misalignment and long reaction time. In this paper, we propose Jetson-PI, a method for efficient VLA deployment on onboard devices via Foresight-Aligned Asynchronous Correction. To address misalignment, we train a lightweight future correction module that predicts future environment representation conditioned on committed actions, enabling the action expert to directly predict actions from the future time step. To reduce reaction time, we introduce confidence-based scheduling optimization that adaptively balances VLM and action expert invocations, complemented by system-level accelerations including CUDA graph reuse, GPU-resident intermediate buffering, and flow unrolling. Extensive experiments demonstrate that Jetson-PI achieves 8.66x and 5.41x improvements in control frequency compared with naive PyTorch and vla.cpp on NVIDIA Jetson Orin, while outperforming VLASH by 14.8\% in average success rate on the LIBERO benchmark. The code of our asynchronous algorithm is available on https://github.com/PKU-SEC-Lab/Jetson-PI, and our efficient llama.cpp-based inference engine is available on https://github.com/PKU-SEC-Lab/Jetson-PI-Edge.

Problem

• Vision-Language-Action models
• onboard deployment
• inference latency
• asynchronous inference
• control frequency

Innovation

• Foresight-Aligned Asynchronous Inference
• Vision-Language-Action Models
• Onboard Real-Time Control
• Confidence-Based Scheduling
• System-Level Acceleration