LA4VLA: Learning to Act without Seeing via Language-Action Pretraining

2026-06-25

AI Summary

Current vision-language-action (VLA) models overly rely on visual inputs and underutilize the conditioning role of language in action generation, leading to poor generalization under visual domain shifts. To address this limitation, this work proposes LA4VLA, a novel framework that introduces, for the first time, a vision-free language-action (LA) pretraining paradigm. It leverages three strategies—LA-only, sequential, and mixed—to learn language-conditioned action priors from a newly curated LA-33K dataset, subsequently fusing these priors with visual information to enhance policy robustness. Experimental results demonstrate that the mixed LA-VLA pretraining strategy significantly outperforms conventional VLA baselines, improving task success rates by 17.8 and 45.0 percentage points in simulated and real-world settings, respectively.

Abstract

Vision-Language-Action (VLA) models are commonly pretrained on robot demonstrations by jointly mapping visual observations and language instructions to actions. However, dense visual-action supervision can dominate the comparatively sparse language-action signal. As a result, policies may rely on visual shortcuts rather than learn how language conditions action execution, making them sensitive to visual variations. To address this limitation, we propose LA4VLA, a language-action pretraining framework that enables policies to acquire language-conditioned action priors without visual observations. These priors capture reusable manipulation skills shared across tasks and scenes, reducing reliance on scene-specific visual cues. Specifically, LA4VLA decomposes expert demonstration trajectories into atomic action segments and pairs each segment with a corresponding low-level action description. This yields LA4-33K, a dataset of 33K Language-Action (LA) episodes derived entirely from existing demonstrations without additional robot data collection. We further develop LA4VLA-1B, a lightweight 1B-parameter VLA model, and investigate three paradigms for incorporating language-action supervision into VLA learning: LA-only pretraining, sequential LA-to-VLA pretraining, and mixed LA-VLA pretraining. Across simulation and real-world tasks, LA-pretrained policies consistently outperform matched VLA-pretrained counterparts, while combining LA and VLA supervision leads to further gains. In particular, mixed LA-VLA pretraining improves the average success rate of LA4VLA-1B over the no-pretraining baseline by up to 17.8 and 45.0 percentage points in simulation and real-world tasks, respectively. These results establish LA4VLA as an effective and complementary pretraining strategy for building stronger and more robust VLA policies.

Problem

• Vision-Language-Action
• language-conditioned action
• visual shortcuts
• action priors
• robot learning

Innovation

• Language-Action Pretraining
• Vision-Language-Action Models
• Action Priors
• Robotic Policy Learning
• Visual Shortcut Mitigation