Implicit Kinodynamic Motion Retargeting for Human-to-humanoid Imitation Learning

2025-09-18

AI Summary

This work addresses the challenge of efficiently and physically plausibly transferring large-scale human motion to full-body trajectories for humanoid robots. We propose an implicit dynamics-aware motion retargeting framework that jointly encodes motion topology and robot dynamics. Methodologically, we introduce a dual-encoder–decoder architecture, integrating motion representation learning, imitation learning, and retargeting into a single end-to-end trainable model—marking the first such unified approach. The framework jointly enforces kinematic constraints and dynamic feasibility, enabling real-time, scalable whole-body motion transfer. Evaluated in simulation and on physical full-size humanoid robots, our method generates trajectories that directly drive low-level whole-body controllers, achieving high-fidelity motion tracking. It significantly outperforms conventional frame-wise retargeting techniques in accuracy, robustness, and execution reliability. This establishes a new paradigm for efficient, physics-aware motion transfer in embodied agents, advancing the deployment of learned human behaviors onto complex robotic platforms.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Motion & Tracking

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Human-to-humanoid imitation learning aims to learn a humanoid whole-body controller from human motion. Motion retargeting is a crucial step in enabling robots to acquire reference trajectories when exploring locomotion skills. However, current methods focus on motion retargeting frame by frame, which lacks scalability. Could we directly convert large-scale human motion into robot-executable motion through a more efficient approach? To address this issue, we propose Implicit Kinodynamic Motion Retargeting (IKMR), a novel efficient and scalable retargeting framework that considers both kinematics and dynamics. In kinematics, IKMR pretrains motion topology feature representation and a dual encoder-decoder architecture to learn a motion domain mapping. In dynamics, IKMR integrates imitation learning with the motion retargeting network to refine motion into physically feasible trajectories. After fine-tuning using the tracking results, IKMR can achieve large-scale physically feasible motion retargeting in real time, and a whole-body controller could be directly trained and deployed for tracking its retargeted trajectories. We conduct our experiments both in the simulator and the real robot on a full-size humanoid robot. Extensive experiments and evaluation results verify the effectiveness of our proposed framework.

Problem

• Efficient human-to-humanoid motion retargeting for imitation learning
• Converting human motion into physically feasible robot trajectories
• Real-time scalable motion retargeting considering kinematics and dynamics

Innovation

• Implicit Kinodynamic Motion Retargeting framework
• Dual encoder-decoder architecture mapping
• Integrates imitation learning with dynamics