UniT: Toward a Unified Physical Language for Human-to-Humanoid Policy Learning and World Modeling

2026-04-21

AI Summary

This work addresses the challenge of cross-embodiment policy transfer arising from scarce robot data and kinematic discrepancies between humans and humanoid robots. To this end, the authors propose UniT, a framework that constructs a unified physical language through visual anchoring and employs a three-branch cross-reconstruction architecture to align human motions with their visual outcomes. This alignment yields an embodiment-agnostic, discrete latent intention space, establishing—for the first time—a shared physical intention representation across embodiments. The framework enables zero-shot task transfer from human videos to humanoid robots and supports controllable motion video generation. Experiments demonstrate that UniT achieves state-of-the-art data efficiency and out-of-distribution generalization in both simulation and real-world environments.

Technology Category

• Intelligent Robots: Embodied AI
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Multi-modal Vision

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Scaling humanoid foundation models is bottlenecked by the scarcity of robotic data. While massive egocentric human data offers a scalable alternative, bridging the cross-embodiment chasm remains a fundamental challenge due to kinematic mismatches. We introduce UniT (Unified Latent Action Tokenizer via Visual Anchoring), a framework that establishes a unified physical language for human-to-humanoid transfer. Grounded in the philosophy that heterogeneous kinematics share universal visual consequences, UniT employs a tri-branch cross-reconstruction mechanism: actions predict vision to anchor kinematics to physical outcomes, while vision reconstructs actions to filter out irrelevant visual confounders. Concurrently, a fusion branch synergies these purified modalities into a shared discrete latent space of embodiment-agnostic physical intents. We validate UniT across two paradigms: 1) Policy Learning (VLA-UniT): By predicting these unified tokens, it effectively leverages diverse human data to achieve state-of-the-art data efficiency and robust out-of-distribution (OOD) generalization on both humanoid simulation benchmark and real-world deployments, notably demonstrating zero-shot task transfer. 2) World Modeling (WM-UniT): By aligning cross-embodiment dynamics via unified tokens as conditions, it realizes direct human-to-humanoid action transfer. This alignment ensures that human data seamlessly translates into enhanced action controllability for humanoid video generation. Ultimately, by inducing a highly aligned cross-embodiment representation (empirically verified by t-SNE visualizations revealing the convergence of human and humanoid features into a shared manifold), UniT offers a scalable path to distill vast human knowledge into general-purpose humanoid capabilities.

Problem

• humanoid
• cross-embodiment
• kinematic mismatch
• policy learning
• world modeling

Innovation

• cross-embodiment transfer
• unified physical language
• latent action tokenization
• visual anchoring
• humanoid foundation models