Spatio-Temporal Motion Retargeting for Quadruped Robots

2024-04-17

arXiv.org

AI Summary

This work addresses the challenge of enabling quadrupedal robots to dynamically imitate animal locomotion across morphologically distinct bodies. We propose a Spatio-Temporal Motion Retargeting (STMR) framework that decouples Spatial Motion Retargeting (SMR) and Temporal Motion Retargeting (TMR), respectively ensuring kinematic feasibility and dynamic executability—overcoming the failure of conventional methods on highly dynamic aerial phases. The method integrates keypoint trajectory mapping, full-body inverse kinematics, temporal-domain optimization, and end-to-end imitation learning, supporting monocular video input and real-hardware deployment. Evaluated in simulation and on two physically distinct quadruped platforms, STMR successfully reproduces six complex animal gaits with high-fidelity trajectory tracking. Quantitative results demonstrate significant performance gains over baseline approaches in both accuracy and dynamic consistency.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Computer Vision: Motion & Tracking

Application Category

• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

This work introduces a motion retargeting approach for legged robots, which aims to create motion controllers that imitate the fine behavior of animals. Our approach, namely spatio-temporal motion retargeting (STMR), guides imitation learning procedures by transferring motion from source to target, effectively bridging the morphological disparities by ensuring the feasibility of imitation on the target system. Our STMR method comprises two components: spatial motion retargeting (SMR) and temporal motion retargeting (TMR). On the one hand, SMR tackles motion retargeting at the kinematic level by generating kinematically feasible whole-body motions from keypoint trajectories. On the other hand, TMR aims to retarget motion at the dynamic level by optimizing motion in the temporal domain. We showcase the effectiveness of our method in facilitating Imitation Learning (IL) for complex animal movements through a series of simulation and hardware experiments. In these experiments, our STMR method successfully tailored complex animal motions from various media, including video captured by a hand-held camera, to fit the morphology and physical properties of the target robots. This enabled RL policy training for precise motion tracking, while baseline methods struggled with highly dynamic motion involving flying phases. Moreover, we validated that the control policy can successfully imitate six different motions in two quadruped robots with different dimensions and physical properties in real-world settings.

Problem

• Transfer dynamic motions to robots despite morphological differences
• Ensure physical feasibility in motion retargeting for legged robots
• Convert noisy motion sources into robot-specific feasible movements

Innovation

• Motion retargeting bridges morphological disparities effectively
• Kinematic and dynamic refinement ensures physical feasibility
• Policy training via reinforcement learning enables robust tracking