HumanoidUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation

2026-06-25

AI Summary

This work addresses the challenge of acquiring whole-body manipulation demonstrations for humanoid robots, which is hindered by the high cost, skill requirements, and inefficiency of existing teleoperation systems. The authors propose a portable, robot-free data collection framework that leverages lightweight VR hardware and a UMI-inspired gripper to simultaneously capture sparse human body keypoints, wrist-view images, and gripper actions. A high-level policy predicts future keypoints and retargets them into full-body reference commands for a whole-body controller to execute. This approach enables, for the first time, the collection of whole-body manipulation demonstrations without requiring a physical robot, extending the UMI paradigm to full-body behaviors and supporting cross-task skill transfer. The effectiveness and generalizability of the learned skills are validated across five real-world scenarios.

Abstract

High-quality demonstration data are essential for humanoid robot skill learning, especially for whole-body behaviors that require coordinated perception, locomotion, and manipulation. Existing data-collection methods largely rely on robot teleoperation, which is constrained by hardware accessibility, operator expertise, and limited efficiency. Inspired by the Universal Manipulation Interface (UMI), we propose HumanoidUMI, a portable and robot-free framework for humanoid whole-body data collection. HumanoidUMI uses lightweight VR devices and UMI-inspired grippers to collect sparse human keypoint trajectories, wrist-view observations, and gripper actions. These demonstrations train a high-level policy to predict future keypoints, which are retargeted to robot-native whole-body references and executed by a whole-body controller. Experiments in five real-world scenarios demonstrate the effectiveness of the proposed framework and validate the collected demonstrations for transferable humanoid whole-body skill learning.

Problem

• humanoid robots
• demonstration data
• whole-body manipulation
• teleoperation
• skill learning

Innovation

• HumanoidUMI
• robot-free demonstration
• whole-body manipulation
• keypoint retargeting
• VR-based data collection