Composing Dextrous Grasping and In-hand Manipulation via Scoring with a Reinforcement Learning Critic

2025-05-19

AI Summary

Dexterous robotic grasping and in-hand manipulation have traditionally been decoupled, necessitating manual pre-placement of stable initial grasps and hindering end-to-end autonomy. Method: We propose a unified optimization framework that for the first time directly transfers the critic network—trained via Soft Actor-Critic (SAC) for in-hand manipulation—to evaluate grasp pose quality, jointly modeling both grasp stability and task-directed manipulation objectives without additional training. Based on critic scores, we perform efficient grasp sampling, ranking, and selection, and integrate real-time closed-loop in-hand reorientation control. Contribution/Results: Our approach breaks the conventional two-stage paradigm, enabling fully autonomous, task-directed grasping and reorientation of irregular, challenging objects on a physical dexterous hand, significantly improving manipulation success rates.

Technology Category

• Intelligent Robots: Manipulation
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

In-hand manipulation and grasping are fundamental yet often separately addressed tasks in robotics. For deriving in-hand manipulation policies, reinforcement learning has recently shown great success. However, the derived controllers are not yet useful in real-world scenarios because they often require a human operator to place the objects in suitable initial (grasping) states. Finding stable grasps that also promote the desired in-hand manipulation goal is an open problem. In this work, we propose a method for bridging this gap by leveraging the critic network of a reinforcement learning agent trained for in-hand manipulation to score and select initial grasps. Our experiments show that this method significantly increases the success rate of in-hand manipulation without requiring additional training. We also present an implementation of a full grasp manipulation pipeline on a real-world system, enabling autonomous grasping and reorientation even of unwieldy objects.

Problem

• Bridging gap between grasping and in-hand manipulation in robotics
• Selecting initial grasps that enable successful in-hand manipulation
• Enabling autonomous grasping and reorientation of unwieldy objects

Innovation

• Uses RL critic to score initial grasps
• Enables autonomous grasp and reorientation
• Improves success rate without extra training