RoVLA: Multi-Consistency Constraints for Robust Vision-Language-Action Models

2026-05-19

AI Summary

Current vision-language-action (VLA) models exhibit fragility under visual perturbations, instruction rephrasing, and composite disturbances, often over-relying on superficial correlations present in the training distribution. This work proposes RoVLA, a novel framework that, for the first time, integrates triple consistency constraints—spanning instruction semantics, trajectory evolution, and observation perturbations—within an end-to-end VLA architecture to explicitly model the stable coupling among task semantics, environmental states, and action generation. Trained end-to-end with a multi-consistency loss function, RoVLA substantially outperforms existing baselines on LIBERO-Plus, RoboTwin 2.0, and real-world robotic tasks, demonstrating exceptional robustness and generalization across diverse tasks and observation shifts.

Abstract

Vision-Language-Action (VLA) models have shown strong performance on embodied manipulation, yet they remain brittle under visual observation changes, paraphrased language instructions, and compounded perturbations. This limitation suggests that existing methods still rely heavily on shallow correlations in the training distribution, rather than learning stable couplings among task semantics, environment states, and action generation. Although recent efforts improve robustness through larger-scale training, post-training adaptation, or enhanced predictive modeling, they rarely enforce invariance-oriented consistency within the end-to-end policy itself. To address this issue, we propose RoVLA, a robust vision-language-action framework with multi-consistency constraints. RoVLA enforces consistency under three complementary transformations: instruction semantics, trajectory evolution, and observation perturbation. Specifically, Instructional Consistency (IC) promotes stable grounding under semantically equivalent instruction rewrites, Evolutionary Consistency (EC) preserves coherent action intent throughout the generation process, and Observational Consistency (OC) improves robustness to visual and proprioceptive perturbations by enforcing consistent predictions before and after targeted disturbances. By explicitly modeling these invariances during training, RoVLA reduces reliance on superficial correlations and improves robustness and generalization. Experiments on LIBERO-Plus, RoboTwin 2.0, and real-world manipulation tasks show that RoVLA consistently outperforms strong baseline methods and exhibits superior robustness under diverse task and observation shifts. These results demonstrate the effectiveness of multi-consistency learning for robust embodied control. Codes will be available at https://github.com/HCPLab-SYSU/RoVLA.

Problem

• Vision-Language-Action
• robustness
• consistency constraints
• embodied manipulation
• distribution shift

Innovation

• multi-consistency constraints
• vision-language-action models
• robust embodied control
• instructional consistency
• observational robustness