CoRE-VLA: Towards Scalable and Robust Vision-Language-Action Modeling via Conditional Routing of Experts

2026-07-03

AI Summary

This work addresses the limited robustness of existing vision-language-action (VLA) models under missing or heterogeneous sensor configurations, which hinders their deployment in real-world robotic settings. The authors propose CoRE-VLA, a novel framework that introduces conditional sparse expert routing into VLA modeling for the first time. By leveraging a dynamic routing mechanism jointly driven by sensor gating and task intent, CoRE-VLA enables context-aware sparse computation and supports graceful degradation under sensor loss without requiring retraining. The method substantially improves multi-task generalization and long-horizon behavioral stability, outperforming dense baselines and state-of-the-art pretrained VLA models across LIBERO, RoboCasa GR1, and a real dual-arm platform—particularly excelling in zero-shot and sensor-deprived scenarios with remarkable performance and robustness.

Abstract

Vision-language-action (VLA) models have advanced generalist robotic manipulation, yet real-world deployment reveals a fundamental challenge: robots are equipped with diverse and heterogeneous sensor configurations, auxiliary sensors can fail unexpectedly during operation, and different robot embodiments often lack certain sensors by design. A unified policy that can exploit auxiliary perceptual inputs when available while remaining reliable under sensor absence, whether incidental or by design, is therefore essential for practical deployment. However, existing VLA policies couple action generation to a fixed sensor set through shared dense computation, making them brittle when sensors are missing and limiting their ability to specialize across diverse tasks and long-horizon behaviors. We propose CoRE-VLA, a scalable and robust VLA framework that formulates action generation as context-conditioned sparse computation. Sensor availability gates modality-specialized experts, enabling graceful degradation under missing sensors without retraining. Task intent further routes action-side representations to task-relevant experts, improving specialization across diverse tasks and long-horizon subgoals. While the framework is designed to accommodate different auxiliary sensors, we focus on depth as a representative and practically important auxiliary modality in our experiments. Experiments on LIBERO, RoboCasa GR1 Tabletop, and real-world dual-arm manipulation show that CoRE-VLA achieves strong results on long-horizon and multi-task benchmarks, and outperforms both a dense-action-generator ablation and a strong pretrained VLA baseline, including in zero-shot generalization to unseen scenarios. Modality analysis shows that CoRE-VLA can exploit auxiliary depth when available while remaining robust when depth is unavailable during deployment.

Problem

• vision-language-action
• sensor heterogeneity
• sensor failure
• robustness
• scalability

Innovation

• Conditional Routing
• Sparse Computation
• Modality Robustness
• Vision-Language-Action Models
• Graceful Degradation