WOLF-VLA: Whole-Body Humanoid Optimal Locomotion Framework for Vision-Language-Action Learning

2026-06-24

AI Summary

This work addresses key challenges in applying vision–language–action (VLA) models to whole-body, contact-rich humanoid robot control—namely, data scarcity, dynamically inconsistent demonstrations, and the difficulty of simultaneously ensuring optimality and safety. To overcome these limitations, we propose an end-to-end learning framework that integrates whole-body optimal control with large-scale multimodal data. We introduce, for the first time, a dynamically consistent multimodal dataset of whole-body humanoid motions, enabling direct generation of robust, safe, and high-performance motor policies from natural language instructions. Experimental results demonstrate that the learned policies exhibit strong generalization across diverse tasks and environmental conditions, robustness to variations in initial states, and state-of-the-art performance across multiple metrics, while establishing a reproducible benchmark for future research.

Abstract

Vision-Language-Action (VLA) models have recently demonstrated strong generalization in robotic manipulation, yet their applicability to whole-body, contact-rich humanoid locomotion remains severely underexplored due to data scarcity, the absence of dynamically consistent demonstrations, and the difficulty of encoding optimality and safety in learning-based pipelines. This work introduces a unified framework WOLF-VLA that integrates whole-body optimal-control (OC) motion synthesis with large-scale multi-modal dataset to train VLAs capable of generating humanoid locomotion policies directly from natural-language instructions. We construct a comprehensive dataset of dynamically feasible humanoid trajectories across six locomotion-related task families, each parameterized by environmental variations, object colors, placements, and visual distractors. We train a VLA model using the collected joint trajectories, ego-centric visual observations and natural language instruction, yielding a policy that exhibits strong reasoning and robustness to initial-condition variability, and competitive performance across several tasks and environment settings. A systematic ablation study demonstrates the impact of each modality on the model performance. The full dataset, model checkpoints, and benchmarking simulation suite will be openly released, establishing a reproducible dynamically consistent benchmark for whole-body humanoid locomotion rich VLA control and enabling future research in scalable transfer of instruction-driven locomotion policies.

Problem

• Vision-Language-Action
• humanoid locomotion
• data scarcity
• dynamically consistent demonstrations
• whole-body control

Innovation

• Vision-Language-Action
• Whole-Body Locomotion
• Optimal Control
• Humanoid Robotics
• Multimodal Learning