Semi-Supervised Vision-Language-Action Model

2026-06-19

AI Summary

This work addresses the challenge of adapting vision–language–action (VLA) models to new environments under limited action annotations by proposing SemiVLA, a semi-supervised framework that operates with only a small fraction of labeled trajectories. Built upon a teacher–student self-distillation architecture, SemiVLA generates pseudo-actions from unlabeled trajectories and introduces a novel multi-dimensional reliability controller that jointly assesses visual–language alignment, action feasibility, and temporal consistency. To mitigate noise propagation, it further incorporates a Bottleneck-Projected Alignment Update mechanism that safeguards the teacher model from contamination by unreliable feedback. Leveraging the OpenVLA backbone with parameter-efficient fine-tuning, SemiVLA achieves an average success rate of 89.0% on the LIBERO benchmark using just 10% of labeled data—surpassing fully supervised LoRA by 8.0 percentage points—without incurring additional inference overhead, and demonstrates consistent superiority over baselines on CALVIN as well.

Abstract

Vision-Language-Action (VLA) models enable robots to predict actions directly from visual observations and language instructions, but adapting them to new environments still depends on costly action-labeled demonstrations. To reduce this dependence, we study semi-supervised VLA adaptation under limited supervision signals, where only a small portion of trajectories contain robot actions and the remaining trajectories provide action-unlabeled vision-language observations. Unlike standard semi-supervised learning, the missing supervision is an embodied action signal that must be visually grounded, language-consistent, physically feasible, and temporally stable. To address this problem, we propose SemiVLA, a self-distilled teacher-student framework that learns from reliable pseudo-actions on unlabeled trajectories. SemiVLA introduces a VLA-specific reliability controller to assess vision-language alignment, action feasibility, and temporal transition consistency, and further updates the teacher through a Bottleneck-Projected Alignment Update to avoid noisy feedback contamination. With OpenVLA as the backbone, SemiVLA consistently improves multiple PEFT strategies across LIBERO and CALVIN. Under 10\% labeled trajectories, SemiVLA with Selective LoRA achieves 89.0\% average success on LIBERO, outperforming supervised LoRA by 8.0 points without extra inference cost.

Problem

• Semi-Supervised Learning
• Vision-Language-Action Models
• Robot Action Adaptation
• Limited Supervision
• Embodied Action Grounding

Innovation

• Semi-Supervised Learning
• Vision-Language-Action Model
• Self-Distillation
• Pseudo-Action Reliability
• Bottleneck-Projected Alignment