Learning Cross-hand Policies for High-DOF Reaching and Grasping

2024-04-14

arXiv.org

AI Summary

Cross-platform reuse of dexterous hand grasping policies remains challenging due to hardware-specific hand morphology and control interfaces. Method: This paper proposes a hand-agnostic, two-stage unified framework: (1) predicting displacement vectors of object surface key points—decoupled from hand anatomy—and (2) mapping these predictions to target hand joint controls via a differentiable adaptation module. We introduce finger-level geometric representations to model hand-object interactions and integrate Transformers to handle structural heterogeneity across diverse dexterous hands. A hierarchical strategy design enables decoupling of perception and control, while end-to-end differentiability supports zero-shot or few-shot transfer. Results: Extensive experiments on multiple high-DOF dexterous hands and complex objects demonstrate significant improvements over state-of-the-art baselines, validating strong generalization across hand morphologies and robust cross-hardware deployability.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Biometrics, Face, Gesture & Pose
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance

Abstract

Reaching-and-grasping is a fundamental skill for robotic manipulation, but existing methods usually train models on a specific gripper and cannot be reused on another gripper. In this paper, we propose a novel method that can learn a unified policy model that can be easily transferred to different dexterous grippers. Our method consists of two stages: a gripper-agnostic policy model that predicts the displacements of pre-defined key points on the gripper, and a gripper-specific adaptation model that translates these displacements into adjustments for controlling the grippers' joints. The gripper state and interactions with objects are captured at the finger level using robust geometric representations, integrated with a transformer-based network to address variations in gripper morphology and geometry. In the experiments, we evaluate our method on several dexterous grippers and diverse objects, and the result shows that our method significantly outperforms the baseline methods. Pioneering the transfer of grasp policies across dexterous grippers, our method effectively demonstrates its potential for learning generalizable and transferable manipulation skills for various robotic hands.

Problem

• Robot Learning
• Universal Gripping Skills
• Adaptability to Different Hand Types

Innovation

• Two-step Training Strategy
• Dexterous Hand Adaptability
• Specialized Network for Hand Diversity