FutureVLA: Joint Visuomotor Prediction for Vision-Language-Action Model

2026-03-11

AI Summary

Existing vision-language-action (VLA) models struggle to jointly model visual and action modalities while preserving temporal continuity and respecting environmental constraints, often being misled by static scene elements and overlooking dynamic intent. To address this, this work proposes FutureVLA, which decouples visual and action representations through a joint visuomotor gating mechanism and a latent embedding alignment strategy. During pretraining, the model separates visual state retention from action modeling and incorporates physical priors to enhance temporal consistency. Trained on heterogeneous manipulation datasets, FutureVLA produces highly generalizable joint visuomotor embeddings, significantly improving action prediction accuracy and environmental interaction capabilities across diverse VLA frameworks in downstream tasks.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Predictive foresight is important to intelligent embodied agents. Since the motor execution of a robot is intrinsically constrained by its visual perception of environmental geometry, effectively anticipating the future requires capturing this tightly coupled visuomotor interplay. While recent vision-language-action models attempt to incorporate future guidance, they struggle with this joint modeling. Existing explicit methods divert capacity to task-irrelevant visual details, whereas implicit methods relying on sparse frame pairs disrupt temporal continuity. By heavily relying on visual reconstruction, these methods become visually dominated, entangling static scene context with dynamic action intent. We argue that effective joint visuomotor predictive modeling requires both temporal continuity and visually-conditioned supervision decoupling. To this end, we propose FutureVLA, featuring a novel Joint Visuomotor Predictive Architecture. FutureVLA is designed to extract joint visuomotor embeddings by first decoupling visual and motor information, and then jointly encoding generalized physical priors. Specifically, in the pretraining stage, we leverage heterogeneous manipulation datasets and introduce a Joint Visuomotor Gating mechanism to structurally separate visual state preservation from temporal action modeling. It allows the motor stream to focus on continuous physical dynamics while explicitly querying visual tokens for environmental constraints, yielding highly generalizable joint visuomotor embeddings. Subsequently, in the post-training stage, we employ a latent embeddings alignment strategy, enabling diverse downstream VLA models to internalize these temporal priors without modifying their inference architectures. Extensive experiments demonstrate that FutureVLA consistently improves VLA frameworks.

Problem

• visuomotor prediction
• vision-language-action model
• temporal continuity
• visual-motor decoupling
• predictive foresight

Innovation

• Joint Visuomotor Prediction
• Temporal Continuity
• Visuomotor Decoupling
• Gating Mechanism
• Latent Embedding Alignment