OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interaction

2025-09-30

AI Summary

Existing motion retargeting methods struggle with morphological discrepancies between humans and robots, leading to physical implausibilities (e.g., foot sliding, interpenetration) and neglecting human–object–environment interactions—thus limiting expressive locomotion and mobile manipulation. This paper proposes an interaction-preserving motion retargeting framework: it constructs an interaction mesh to explicitly model contact states and spatial relationships among the human body, objects, and environment; integrates Laplacian deformation minimization with kinematic constraints to generate high-fidelity, physically plausible full-body trajectories; and enables data augmentation from a single demonstration to diverse robot morphologies, terrains, and object configurations. Evaluated on multiple motion-capture datasets, the method synthesizes over eight hours of high-quality trajectories, significantly outperforming baselines. Using only five sparse rewards, it successfully trains a Unitree G1 quadruped to complete 30-second parkour and manipulation tasks.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Motion & Tracking
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search

Abstract

A dominant paradigm for teaching humanoid robots complex skills is to retarget human motions as kinematic references to train reinforcement learning (RL) policies. However, existing retargeting pipelines often struggle with the significant embodiment gap between humans and robots, producing physically implausible artifacts like foot-skating and penetration. More importantly, common retargeting methods neglect the rich human-object and human-environment interactions essential for expressive locomotion and loco-manipulation. To address this, we introduce OmniRetarget, an interaction-preserving data generation engine based on an interaction mesh that explicitly models and preserves the crucial spatial and contact relationships between an agent, the terrain, and manipulated objects. By minimizing the Laplacian deformation between the human and robot meshes while enforcing kinematic constraints, OmniRetarget generates kinematically feasible trajectories. Moreover, preserving task-relevant interactions enables efficient data augmentation, from a single demonstration to different robot embodiments, terrains, and object configurations. We comprehensively evaluate OmniRetarget by retargeting motions from OMOMO, LAFAN1, and our in-house MoCap datasets, generating over 8-hour trajectories that achieve better kinematic constraint satisfaction and contact preservation than widely used baselines. Such high-quality data enables proprioceptive RL policies to successfully execute long-horizon (up to 30 seconds) parkour and loco-manipulation skills on a Unitree G1 humanoid, trained with only 5 reward terms and simple domain randomization shared by all tasks, without any learning curriculum.

Problem

• Generating physically plausible humanoid motion trajectories from human demonstrations
• Preserving human-object and human-environment interactions during motion retargeting
• Addressing embodiment gaps between humans and robots in motion data

Innovation

• Interaction mesh preserves spatial and contact relationships
• Minimizes Laplacian deformation with kinematic constraints
• Enables data augmentation across robot embodiments and environments