Human-as-Humanoid: Enabling Zero-Shot Humanoid Learning from Ego-Exo Human Videos with Human-Aligned Embodiments

2026-06-30

AI Summary

This work addresses the challenge of enabling high-degree-of-freedom humanoid robots to learn executable policies directly from human videos, which is hindered by the scarcity of scalable, high-quality observation-action supervision data. The authors propose a zero-shot transfer method that requires no robot demonstrations: by synchronizing egocentric and exocentric video streams, they leverage human motion reconstruction and a staged inverse kinematics retargeting pipeline to accurately map human motions onto a 60-degree-of-freedom upper-body robot. To preserve task-space geometric consistency, the approach incorporates an embodiment-aware alignment design and forward kinematics–aware supervision. The data collection efficiency of this method surpasses teleoperation by 4.8–7.2×, and a vision-language-action model trained solely on the translated human labels successfully performs diverse downstream tasks on a real robot in a zero-shot manner.

Abstract

Vision-language-action (VLA) models across robot embodiments require high-quality observation--action supervision to learn deployable action distributions, yet scaling such robot data remains difficult, especially for high-DoF humanoids. Teleoperation provides controller-aligned supervision, while human egocentric videos capture diverse bimanual manipulation but do not directly provide executable robot actions. We introduce Human-as-Humanoid, a human-to-humanoid supervision framework that enables near-real-time human-centric action generation, making human demonstrations usable for high-DoF humanoid VLA training by jointly aligning the robot embodiment, the sensing setup, and the action-label interface. Built on PrimeU, a human-aligned 60-DoF upper-body humanoid, Human-as-Humanoid uses synchronized ego-exo videos to pair deployment-aligned egocentric observations with exocentric motion recovery, retargets the recovered human motion through staged Inverse Kinematics (IK) into controller-aligned 60-DoF action chunks, and trains the VLA model with Forward Kinematics (FK)-aware supervision to preserve wrist and fingertip task-space geometry. This converts large-scale human demonstrations from visual observations into executable observation--action supervision for the target humanoid. Experiments validate the conversion chain at the motion-recovery, robot-action-space, and real-robot deployment levels. Human-as-Humanoid yields a 4.8--7.2x raw demonstration-throughput gain over humanoid teleoperation in our data-collection analysis, and on several downstream tasks, policies post-trained only with the converted human labels generalize to real-robot deployment without target-task robot demonstrations. The official project website is available at https://zgc-embodyai.github.io/Human-as-Humanoid.

Problem

• humanoid learning
• zero-shot
• ego-exo videos
• action supervision
• embodiment alignment

Innovation

• Human-as-Humanoid
• Vision-Language-Action (VLA)
• Inverse Kinematics (IK)
• Ego-Exo Video Alignment
• Zero-Shot Humanoid Learning