Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos

2026-05-22

AI Summary

This work addresses the challenge of motion transfer in monocular video-driven humanoid robot imitation learning, where morphological discrepancies hinder effective policy generalization. To overcome this, the authors propose Direct Dynamic Retargeting (DDR), a novel single-stage, end-to-end framework that bypasses conventional multi-stage pipelines and intermediate kinematic projections. DDR operates directly in task space by integrating sampling-based model predictive control with physics simulation to generate dynamically feasible and high-fidelity trajectories. The approach natively optimizes complex contact sequences, effectively mitigating input drift and circumventing limitations imposed by restricted search spaces. Experimental results demonstrate that DDR achieves superior demonstration tracking accuracy compared to state-of-the-art methods and provides high-quality reference trajectories for reinforcement learning, significantly accelerating training convergence while enhancing robotic agility and balance.

Abstract

Imitation Learning from monocular video demonstrations provides a scalable approach for teaching complex skills to humanoid robots. However, translating human motion to humanoids requires overcoming significant morphological mismatches. Standard approaches rely on Geometric Retargeting or Indirect Dynamic Retargeting pipelines. We identify that these intermediate kinematic projections introduce a geometric bias, restricting the search space and yielding suboptimal dynamic behaviors. In this paper, we propose Direct Dynamic Retargeting (DDR), a novel single-stage framework that generates high-fidelity, dynamically feasible trajectories directly from expert videos. By formulating the problem in the task space and leveraging a sampling-based Model Predictive Control solver within a physics simulator, DDR natively optimizes over complex contact sequences while mitigating input drift. Our experiments demonstrate that bypassing the geometric bias allows DDR to outperform state-of-the-art baselines in demonstration tracking accuracy. Furthermore, we establish that providing such physically viable references to RL agents accelerates training convergence and enhances the final execution of agile and balancing behaviors. Source code will be made publicly available.

Problem

• Imitation Learning
• Humanoid Robots
• Morphological Mismatch
• Dynamic Retargeting
• Monocular Video

Innovation

• Direct Dynamic Retargeting
• Imitation Learning
• Humanoid Robotics
• Model Predictive Control
• Physics-based Simulation