I-CTRL: Imitation to Control Humanoid Robots Through Constrained Reinforcement Learning

2024-05-14

arXiv.org

AI Summary

This work addresses the core challenge in humanoid robot motion imitation: high visual fidelity versus physical feasibility. We propose a constrained reinforcement learning (CRL) framework that leverages kinematically feasible but dynamically infeasible motion-retargeted trajectories as initial references, then refines them under full physical constraints to achieve high-fidelity, dynamically stable gait transfer. Our method integrates motion retargeting, trajectory-following optimization, and high-fidelity physics simulation (PyBullet/MuJoCo), employing a compact, general-purpose reward function. Crucially, we formulate motion imitation for the first time as constrained fine-tuning of non-physical reference trajectories. Experiments demonstrate significant improvements in tracking accuracy and execution stability across multiple large-scale human motion datasets. Moreover, the framework successfully generalizes to four distinct bipedal robot morphologies, validating its robustness, versatility, and practical applicability.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Motion and Path Planning
• Computer Vision: Motion & Tracking

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

This paper addresses the critical need for refining robot motions that, despite achieving a high visual similarity through human-to-humanoid retargeting methods, fall short of practical execution in the physical realm. Existing techniques in the graphics community often prioritize visual fidelity over physics-based feasibility, posing a significant challenge for deploying bipedal systems in practical applications. Our research introduces a constrained reinforcement learning algorithm to produce physics-based high-quality motion imitation onto legged humanoid robots that enhance motion resemblance while successfully following the reference human trajectory. We name our framework: I-CTRL. By reformulating the motion imitation problem as a constrained refinement over non-physics-based retargeted motions, our framework excels in motion imitation with simple and unique rewards that generalize across four robots. Moreover, our framework can follow large-scale motion datasets with a unique RL agent. The proposed approach signifies a crucial step forward in advancing the control of bipedal robots, emphasizing the importance of aligning visual and physical realism for successful motion imitation.

Problem

• Enhancing humanoid robot motion imitation
• Bridging visual and physical motion realism
• Managing large-scale motion datasets efficiently

Innovation

• Constrained reinforcement learning
• Automatic priority scheduler
• Physics-based motion imitation