Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment

2026-07-15

AI Summary

This work addresses the degradation of general-purpose visual-language representations in vision-language-action (VLA) models during behavioral cloning fine-tuning, which often leads to misalignment between language understanding and action prediction, thereby impairing policy generalization. To mitigate this issue, the authors propose Anchor-Align, a method that simultaneously optimizes language comprehension and action prediction on the same sample through visual-language anchoring and language-action alignment mechanisms. Additionally, it employs layer-wise representation distillation from a frozen copy of the original vision-language model to prevent representational drift. Integrated with behavioral cloning, discrete motion direction supervision, and multi-task training, Anchor-Align substantially improves real-world task success rates on the xArm7 robot—from 28% to 54% and from 37% to 60%—and demonstrates superior out-of-distribution robustness and long-horizon control capabilities across the LIBERO-PRO, LIBERO-Plus, and CALVIN simulation benchmarks.

Abstract

Finetuning a pretrained vision-language model (VLM) on robot demonstrations via behavior cloning (BC) has become the standard recipe for vision-language-action (VLA) policies. However, BC finetuning progressively overwrites the pretrained representations that support visual and semantic generalization. Co-training on web image-text data, a common remedy, does not prevent this; it applies language and action losses to separate observations, leaving VLAs with language-action misalignment that standard manipulation benchmarks do not expose. We propose Anchor-Align, which augments BC with two objectives: Vision-Language Anchoring distills layer-wise representations from a frozen VLM copy to prevent this drift, while Language-Action Alignment converts each action target into a discrete motion-direction label and jointly trains language and action prediction on the same robot observation. On a physical xArm7 robot, across two widely used VLA architectures, Anchor-Align improves real-robot success on both (28% to 54% and 37% to 60%). At scale in simulation, we demonstrate consistent improvements on OOD perturbations, perceptual robustness, and long-horizon control across LIBERO-PRO, LIBERO-Plus, and CALVIN, respectively, suggesting that preserving pretrained representations and effective action learning are not fundamentally at odds. Project page: anchoralignvla.github.io

Problem

• vision-language-action
• behavior cloning
• representation drift
• language-action misalignment
• generalization

Innovation

• Representation Anchoring
• Language-Action Alignment
• Vision-Language-Action Models
• Behavior Cloning
• Generalization