Native Video-Action Pretraining for Generalizable Robot Control

2026-07-09

AI Summary

Existing video generation models struggle to support the semantic action understanding and real-time closed-loop control required for embodied intelligence. To address this limitation, this work proposes LingBot-VA 2.0—the first native video-action foundation model designed specifically for robotic control. The model innovatively integrates a semantically aligned vision-action tokenizer, a causal pretraining paradigm, a sparse mixture-of-experts (MoE) backbone, and an asynchronous parallel inference mechanism grounded in learned forward dynamics. Evaluated in real-world environments, LingBot-VA 2.0 demonstrates strong few-shot generalization on complex manipulation tasks, significantly enhancing both the efficiency and robustness of policy learning, thereby validating its potential as a general-purpose foundation model for robot control.

Abstract

The advent of video-action models offers a promising path for robot control. Nevertheless, we argue that repurposing video generative models designed for digital content creation is inherently inadequate for physical environments. To bridge this gap, we present LingBot-VA 2.0, a video-action foundation model built from the ground up for embodiment. Four core design principles showcase its evolution from LingBot-VA. (1) Departing from traditional reconstruction-focused VAEs, we introduce a semantic visual-action tokenizer, which aligns visual representations with both semantics and actions, improving instruction following and action precision in subsequent policy learning. (2) Given the strictly causal nature of temporal dynamics, we adopt a causal pretraining paradigm, training from scratch to circumvent the catastrophic forgetting that frequently occurs when adapting bidirectional architectures. (3) To meet the demands of high-frequency inference, our model employs a sparse MoE backbone, expanding model capacity without compromising efficiency. (4) Real-time closed-loop control is realized through an enhanced asynchronous inference scheme, which predicts future latents in parallel with action execution while re-grounding each rollout on the latest observation via learned forward dynamics. Real-world deployment validates LingBot-VA 2.0 as a robust foundation model, as evidenced by its few-shot generalization across complex manipulation tasks.

Problem

• video-action models
• robot control
• embodiment
• generalization
• physical environments

Innovation

• video-action foundation model
• semantic visual-action tokenizer
• causal pretraining
• sparse MoE
• asynchronous inference