Task-Oriented Grasping for Dexterous Robots Using Postural Synergies and Reinforcement Learning

2024-12-11

International Conference on Robotic Computing

AI Summary

Existing approaches to humanoid robot grasping struggle to simultaneously satisfy downstream task constraints and human social norms, often lacking end-to-end task-oriented capabilities. This work proposes an end-to-end framework that integrates human grasping preferences with reinforcement learning. It first models hand-pose synergy patterns using the ContactPose dataset and employs a variational autoencoder (VAE) to learn anthropomorphic grasping priors. These priors are then combined with reinforcement learning to optimize task-aware grasping policies. The resulting method enables robots to perform context-aware, ergonomically plausible dexterous grasps on diverse objects according to task intent, significantly enhancing their adaptability and naturalness in human-robot collaborative environments.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Biometrics, Face, Gesture & Pose

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In this paper, we address the problem of task-oriented grasping for humanoid robots, emphasizing the need to align with human social norms and task-specific objectives. Existing methods, employ a variety of open-loop and closed-loop approaches but lack an end-to-end solution that can grasp several objects while taking into account the downstream task's constraints. Our proposed approach employs reinforcement learning to enhance task-oriented grasping, prioritizing the post-grasp intention of the agent. We extract human grasp preferences from the ContactPose dataset, and train a hand synergy model based on the Variational Autoencoder (VAE) to imitate the participant's grasping actions. Based on this data, we train an agent able to grasp multiple objects while taking into account distinct post-grasp intentions that are task-specific. By combining data-driven insights from human grasping behavior with learning by exploration provided by reinforcement learning, we can develop humanoid robots capable of context-aware manipulation actions, facilitating collaboration in human-centered environments.

Problem

• task-oriented grasping
• dexterous robots
• post-grasp intention
• human social norms
• context-aware manipulation

Innovation

• task-oriented grasping
• postural synergies
• reinforcement learning
• human-robot collaboration
• Variational Autoencoder