Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots

2026-07-02

AI Summary

Existing inference runtimes struggle to meet the deployment demands of embodied AI on heterogeneous robotic platforms, particularly regarding multi-rate closed-loop control, low-latency single-sample inference, and extensible interfaces. To address these challenges, this work proposes Embodied.cpp—a lightweight, general-purpose C++-based runtime featuring a five-layer architecture (input adapter, sequence builder, backbone executor, head plugin, and deployment adapter). This design enables modular multi-rate scheduling, low-latency fused inference, and extensible operator and I/O interfaces, thereby transcending the limitations of conventional request-response paradigms. Experimental results demonstrate that the system achieves task success rates of 100.0% and 91.0% on the HY-VLA and pi0.5 models, respectively, while reducing per-module memory consumption from 312.2 MiB to 88.1 MiB on the WAM benchmark.

Abstract

Embodied AI models now span vision-language-action (VLA) models and world-action models (WAMs), but practical deployment remains fragmented across model-specific Python stacks, backend assumptions, and robot-side glue code, especially on heterogeneous edge devices. Existing inference runtimes are designed mainly for request-response serving and therefore do not satisfy the runtime contract of embodied deployment: multi-rate execution inside closed-loop control, latency-first batch-1 inference on heterogeneous hardware, and extensible embodied interfaces beyond fixed token I/O. We present Embodied.cpp, a portable C++ inference runtime for embodied models. Based on an architectural analysis of representative VLA models and WAMs, Embodied.cpp captures a shared execution path and organizes it into five layers: input adapters, sequence builders, backbone execution, head plugins, and deployment adapters. The runtime provides modular multi-rate execution, latency-first fused inference, and extensible operator and I/O support, enabling deployment across heterogeneous devices, robots, and simulators through one backend abstraction. We evaluate Embodied.cpp on two VLA models, HY-VLA and pi0.5, and on a preliminary WAM benchmark using a LingBot-VA Transformer block. The VLA deployments achieve successful closed-loop execution with 100.0% and 91.0% task success rates, respectively. The WAM benchmark reduces block memory from 312.2 MiB to 88.1 MiB. These results show that Embodied.cpp improves deployment efficiency while preserving high accuracy across diverse embodied model architectures.

Problem

• Embodied AI
• heterogeneous robots
• inference runtime
• deployment fragmentation
• closed-loop control

Innovation

• Embodied AI
• inference runtime
• heterogeneous robots
• multi-rate execution
• latency-first inference