InternVLA-A1.5: Unifying Understanding, Latent Foresight, and Action for Compositional Generalization

2026-07-06

AI Summary

This work addresses the challenges in existing unified robotic policy models, which often degrade pretrained semantic representations and suffer from multi-objective interference when integrating visual-language semantics with physical dynamics prediction, while also failing to effectively leverage dynamic priors from pretrained video generation models. The authors propose a novel approach that continually trains a native vision-language model backbone on visual question answering and auxiliary task prediction, introduces a lightweight unified expert module to generate continuous actions, and reformulates future prediction as a latent query problem. Task-relevant dynamic priors are implicitly distilled through learnable foresight tokens under the supervision of a frozen pretrained video generator, avoiding explicit pixel-level prediction. This method achieves the first latent-space integration of video-model dynamics into robotic policy learning, unifying perception, foresight, and action. It attains state-of-the-art performance across six simulation benchmarks and demonstrates superior compositional generalization and long-horizon task execution in real-world settings.

Abstract

Unified models for robot manipulation aim to equip one policy with both the semantic priors of pretrained VLMs and the physical dynamics learned through future prediction. In practice, existing designs tend to erode the semantics of the pretrained backbone, suffer interference among heterogeneous objectives, and learn future prediction from scratch in pixel space, leaving the dynamics priors of pretrained video generators unexploited. We present InternVLA-A1.5, which builds the policy on a native VLM backbone that keeps training on VQA and subtask prediction, and attaches a lightweight unified expert for continuous action generation. Future prediction is recast as a latent-querying problem, where a small set of learnable foresight tokens condenses the task-relevant future into a compact latent code under the supervision of a frozen pretrained video generation model, so the policy inherits world-model dynamics priors without ever learning pixel-level generation. The video branch is discarded at inference, keeping real-time control. Pretrained on 1.2M robot episodes and 3M multimodal samples, InternVLA-A1.5 achieves the best overall results on all six simulation benchmarks. In the real world, the preserved semantics deliver the strongest compositional generalization on held-out instruction bindings, and the two designs together sustain long-horizon execution.

Problem

• compositional generalization
• robot manipulation
• visual-language models
• future prediction
• dynamics priors

Innovation

• latent foresight
• compositional generalization
• unified robot policy
• pretrained VLM backbone
• dynamics priors