ObjRetarget: An Object-Aware Motion Retargeting Framework with Anthropomorphic Arm Constraints and Polyhedral Hand Modeling

2026-07-04

AI Summary

This work addresses the challenges of learning dexterous robotic manipulation from human demonstration videos, particularly the difficulties of intent retargeting and unstable hand–object contacts. The authors propose ObjRetarget, a novel framework that explicitly models multi-finger contacts through polyhedral clusters and integrates anthropomorphic arm motion constraints with geometric invariants to preserve contact structure. By optimizing over redundant degrees of freedom, the method generates natural and executable robot motions. Evaluated on a real robot, ObjRetarget significantly improves success rates and contact stability across diverse dexterous manipulation tasks, demonstrating strong generalization under varying human demonstrations, object poses, and task configurations.

Abstract

Learning robot dexterous manipulation from human manipulation videos requires reliably retargeting human intent to executable robot actions while maintaining stable hand-object contact, which remains a key challenge in embodied intelligence. Existing retargeting methods often ignore explicit contact modeling or rely on reinforcement learning, resulting in limited accuracy and generalization. To address this, we propose ObjRetarget, a human-to-robot motion retargeting framework for learning robot dexterous manipulation from human videos, which integrates anthropomorphic arm trajectory constraints with structured hand-object geometric modeling. For arm motion, reference trajectories extracted from human videos are used for initialization, followed by anthropomorphic constraints and redundancy-aware optimization to generate natural and accurate movements. For hand manipulation, ObjRetarget represents multi-finger contacts using polytope clusters and preserves contact structure through geometric invariants to improve stability. Experiments on real robots show that ObjRetarget improves manipulation success rates and contact stability across multiple dexterous tasks, and generalizes well to different demonstrations, object poses, and task settings.

Problem

• motion retargeting
• dexterous manipulation
• hand-object contact
• embodied intelligence
• human-to-robot

Innovation

• motion retargeting
• polyhedral hand modeling
• anthropomorphic constraints
• hand-object contact
• dexterous manipulation