LIDEA: Human-to-Robot Imitation Learning via Implicit Feature Distillation and Explicit Geometry Alignment

2026-04-12

AI Summary

This work addresses the scarcity of real robot demonstration data and the embodiment gap between human videos and robots in visual appearance and 3D geometry. To bridge this gap, the authors propose the LIDEA framework, which aligns shared 2D visual representations between humans and robots through implicit feature distillation and introduces an embodiment-agnostic explicit 3D geometric alignment strategy to decouple embodiment structure from interaction geometry. By innovatively integrating two-stage transitive distillation, shared latent space modeling, and 3D-aware policy learning, LIDEA is the first method in cross-embodiment imitation learning to simultaneously ensure consistency in 2D representations and invariance in 3D interaction geometry. Experiments demonstrate that the approach achieves baseline-level performance using only 20% of the robot demonstration data and exhibits strong generalization on out-of-distribution tasks.

Technology Category

• Intelligent Robots: Embodied AI
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Computer Vision: Visual Reasoning & Symbolic Representations

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Scaling up robot learning is hindered by the scarcity of robotic demonstrations, whereas human videos offer a vast, untapped source of interaction data. However, bridging the embodiment gap between human hands and robot arms remains a critical challenge. Existing cross-embodiment transfer strategies typically rely on visual editing, but they often introduce visual artifacts due to intrinsic discrepancies in visual appearance and 3D geometry. To address these limitations, we introduce LIDEA (Implicit Feature Distillation and Explicit Geometric Alignment), an imitation learning framework in which policy learning benefits from human demonstrations. In the 2D visual domain, LIDEA employs a dual-stage transitive distillation pipeline that aligns human and robot representations in a shared latent space. In the 3D geometric domain, we propose an embodiment-agnostic alignment strategy that explicitly decouples embodiment from interaction geometry, ensuring consistent 3D-aware perception. Extensive experiments empirically validate LIDEA from two perspectives: data efficiency and OOD robustness. Results show that human data substitutes up to 80% of costly robot demonstrations, and the framework successfully transfers unseen patterns from human videos for out-of-distribution generalization.

Problem

• cross-embodiment transfer
• human-to-robot imitation
• embodiment gap
• robot learning
• visual and geometric discrepancy

Innovation

• implicit feature distillation
• explicit geometry alignment
• cross-embodiment transfer
• imitation learning
• 3D-aware perception