JEPA-VLA: Video Predictive Embedding is Needed for VLA Models

2026-02-12

AI Summary

This work addresses the limitations of existing vision-language-action (VLA) models, which suffer from low sample efficiency and restricted generalization due to their reliance on pretrained visual representations that inadequately capture environmental dynamics and policy priors. To overcome this, we propose JEPA-VLA, a novel framework that, for the first time, identifies and validates the critical role of video prediction embeddings—such as those from V-JEPA 2—in VLA modeling. By introducing an adaptive fusion mechanism, our approach seamlessly integrates self-supervised video-pretrained representations into mainstream VLA architectures, substantially enhancing the model’s ability to capture task-relevant temporal dynamics and learn effective policies. Extensive experiments on LIBERO, LIBERO-plus, RoboTwin2.0, and real-world robotic tasks demonstrate consistent and significant performance improvements, confirming the effectiveness and broad applicability of our method.

Technology Category

• Computer Vision: Large Vision Models
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Recent vision-language-action (VLA) models built upon pretrained vision-language models (VLMs) have achieved significant improvements in robotic manipulation. However, current VLAs still suffer from low sample efficiency and limited generalization. This paper argues that these limitations are closely tied to an overlooked component, pretrained visual representation, which offers insufficient knowledge on both aspects of environment understanding and policy prior. Through an in-depth analysis, we find that commonly used visual representations in VLAs, whether pretrained via language-image contrastive learning or image-based self-supervised learning, remain inadequate at capturing crucial, task-relevant environment information and at inducing effective policy priors, i.e., anticipatory knowledge of how the environment evolves under successful task execution. In contrast, we discover that predictive embeddings pretrained on videos, in particular V-JEPA 2, are adept at flexibly discarding unpredictable environment factors and encoding task-relevant temporal dynamics, thereby effectively compensating for key shortcomings of existing visual representations in VLAs. Building on these observations, we introduce JEPA-VLA, a simple yet effective approach that adaptively integrates predictive embeddings into existing VLAs. Our experiments demonstrate that JEPA-VLA yields substantial performance gains across a range of benchmarks, including LIBERO, LIBERO-plus, RoboTwin2.0, and real-robot tasks.

Problem

• vision-language-action
• visual representation
• sample efficiency
• generalization
• policy prior

Innovation

• predictive embedding
• video pretraining
• vision-language-action
• temporal dynamics
• V-JEPA