SPIDER: Scalable Physics-Informed Dexterous Retargeting

2025-11-12

AI Summary

High-quality demonstration data for dexterous control of humanoid robots and anthropomorphic hands is scarce, while human motion data (e.g., motion capture, video) suffers from embodiment mismatch and lacks dynamical information, hindering direct transfer. Method: We propose SPIDER—the first physics-aware cross-morphological motion retargeting framework. It leverages human motion as global task guidance and integrates curriculum-based virtual contact modeling with large-scale physics simulation sampling to enable end-to-end generation of kinematically consistent and dynamically feasible trajectories. The framework supports multi-source data fusion and morphological generalization. Results: Evaluated across nine robot platforms and six datasets, SPIDER improves task success rates by over 18% compared to state-of-the-art methods, accelerates trajectory generation tenfold relative to reinforcement learning, and synthesizes 2.4 million frames of high-fidelity dynamic training data.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Motion & Tracking

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Learning dexterous and agile policy for humanoid and dexterous hand control requires large-scale demonstrations, but collecting robot-specific data is prohibitively expensive. In contrast, abundant human motion data is readily available from motion capture, videos, and virtual reality, which could help address the data scarcity problem. However, due to the embodiment gap and missing dynamic information like force and torque, these demonstrations cannot be directly executed on robots. To bridge this gap, we propose Scalable Physics-Informed DExterous Retargeting (SPIDER), a physics-based retargeting framework to transform and augment kinematic-only human demonstrations to dynamically feasible robot trajectories at scale. Our key insight is that human demonstrations should provide global task structure and objective, while large-scale physics-based sampling with curriculum-style virtual contact guidance should refine trajectories to ensure dynamical feasibility and correct contact sequences. SPIDER scales across diverse 9 humanoid/dexterous hand embodiments and 6 datasets, improving success rates by 18% compared to standard sampling, while being 10X faster than reinforcement learning (RL) baselines, and enabling the generation of a 2.4M frames dynamic-feasible robot dataset for policy learning. As a universal physics-based retargeting method, SPIDER can work with diverse quality data and generate diverse and high-quality data to enable efficient policy learning with methods like RL.

Problem

• Bridging embodiment gap between human motion and robots
• Converting kinematic human data to dynamic robot trajectories
• Generating scalable physically feasible demonstrations for policy learning

Innovation

• Physics-based retargeting framework transforms human demonstrations
• Scales across diverse humanoid and dexterous hand embodiments
• Generates dynamic-feasible robot trajectories through contact guidance