OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation

2026-06-20

AI Summary

Existing humanoid robots often decouple upper- and lower-body control, hindering coordinated whole-body loco-manipulation. This work proposes the first open-source, whole-body-native vision-language-action (VLA) model that directly maps language instructions and visual inputs to all degrees of freedom in an end-to-end manner. Through staged ablation studies, the authors optimize the teleoperation interface, model architecture, and a heterogeneous co-training strategy. Key findings reveal that joint-level teleoperation outperforms end-effector pose control, models pretrained on static platforms transfer effectively, and heterogeneous co-training generalizes to novel tasks without additional whole-body demonstrations. On challenging long-horizon vertical-space tasks, the proposed approach significantly surpasses state-of-the-art baselines such as GR00T N1.6 and Ψ₀ using less than half the demonstration time.

Abstract

Whole-body humanoid loco-manipulation requires coordinating the robot's entire kinematic chain. However, most existing systems typically decouple the upper and lower bodies into separate controllers, limiting such coordination and yielding behaviors similar to those of a wheeled dual-arm platform. In this paper, we ask what it takes to build a whole-body native vision-language-action (VLA) model that maps language and pixels directly to all of the humanoid's degrees of freedom. We conduct a systematic empirical study organized as a roadmap of one-variable-at-a-time experiments across three phases: whole-body teleoperation, VLA model design, and heterogeneous co-training. Our study yields several intriguing findings: a joint-based whole-body teleoperation interface outperforms alternatives that only partially expose the humanoid's degrees of freedom; a VLA pretrained on static and wheeled dual-arm platforms transfers surprisingly well to a humanoid's full action space; and co-training with HuMI, the humanoid analog of UMI, extends the policy to new objects and instructions without additional whole-body teleoperation on those targets. Following this roadmap yields OpenHLM, an open-source recipe for whole-body humanoid loco-manipulation. In a challenging long-horizon task that spans a wide vertical range of the humanoid, OpenHLM outperforms two state-of-the-art humanoid VLA baselines (GR00T N1.6 and $Ψ_0$) using less than half the total demonstration time. Our code, training data, and model checkpoints are available at [https://openhlm-project.github.io/].

Problem

• humanoid
• loco-manipulation
• whole-body control
• vision-language-action
• teleoperation

Innovation

• whole-body control
• vision-language-action (VLA)
• humanoid locomotion
• teleoperation interface
• heterogeneous co-training