From Grasps to Dexterity: Large-Scale Grasp Pretraining for Dexterous Manipulation

2026-06-29

AI Summary

This work investigates how large-scale grasping data can enhance robotic dexterity in complex tool manipulation tasks, moving beyond their conventional use for basic grasping. The authors construct a pretraining dataset comprising 355,000 grasp trajectories and introduce a hierarchical imitation learning framework: a high-level policy predicts hand subgoals, while a low-level controller executes goal-conditioned actions. By combining pretraining on this dataset with fine-tuning on downstream tasks, the approach enables efficient policy transfer. This study presents the first demonstration that large-scale grasping data can effectively pretrain contact-rich dexterous manipulation policies. The method significantly outperforms both end-to-end diffusion-based strategies and hierarchical baselines trained from scratch on the DexCraft simulation benchmark and in real-world experiments, achieving a 33.3 percentage point improvement in task success rate over DP3.

Abstract

Large-scale dexterous grasp datasets encode rich priors over hand-object interaction, but their use has largely been confined to grasp generation and pick-and-place manipulation. We study whether such data can instead support functional dexterity in articulated tool use, where a robot must acquire a tool, maintain contact, and operate its functional moving parts. We adapt a hierarchical imitation learning framework that combines high-level hand sub-goal prediction with a low-level goal-conditioned controller. We construct a 355k-trajectory grasp-pretraining dataset from large-scale dexterous grasp annotations and use it to pretrain the low-level controller. The controller is then fine-tuned on downstream task demonstrations. To evaluate this setting, we introduce DexCraft, a simulation benchmark with six articulated tool-use tasks requiring coordinated finger motion. Across simulation and real-world experiments, our approach outperforms end-to-end diffusion policy baselines and hierarchical policies trained from scratch. In the real world, it improves full-task success by 33.3 percentage points over DP3. These results show that grasp datasets can serve not only as resources for grasp synthesis, but also as scalable pretraining data for contact-rich dexterous manipulation. Videos are shown on https://yingyuan0414.github.io/grasp2dexterity/ .

Problem

• dexterous manipulation
• grasp pretraining
• articulated tool use
• contact-rich interaction
• functional dexterity

Innovation

• dexterous manipulation
• grasp pretraining
• hierarchical imitation learning
• articulated tool use
• goal-conditioned control