AutoDex: An Automated Real-World System for Dexterous Grasping Data Collection

2026-06-22

AI Summary

This work addresses the challenge of acquiring large-scale, high-quality real-world data for dexterous grasping, where teleoperation is inefficient and biased, while simulation lacks physical validity. The authors propose the first fully autonomous closed-loop system that integrates perception, execution, labeling, and active reset to collect physically grounded grasp data without human intervention. The system employs a dense 20-camera visual setup to handle severe hand-object occlusion, supports interchangeable grasp generators, and uses active resetting to expose a broader set of stable grasp poses. Over 3,593 trials across 100 diverse objects demonstrate a 4.8× speedup in data collection compared to teleoperation. Grasps retrieved from this dataset achieve a 76% success rate, substantially outperforming the 34% success rate of purely simulated approaches.

Abstract

Learning robust dexterous grasping requires real-world data that records the physical outcomes of grasp attempts. Such data is hard to obtain at scale: teleoperation yields valid physical outcomes but is slow and operator-biased, while simulation-based generation is cheap and scalable but cannot certify contact validity. A natural solution is to generate candidate grasps and verify them on real hardware, but this scales only if the entire collection loop (perception, execution, labeling, and reset) runs without human intervention. We present AutoDex, an automated real-world data-collection system that closes this loop: for each candidate from a replaceable generator, it localizes the object under severe hand-object occlusion with dense 20-camera perception, executes collision-monitored robot motions, labels lift-and-hold success or failure, and actively resets the object between trials to expose additional candidates across stable poses. The result is a reusable database of physically labeled grasp trials that downstream systems can query by retrieval and feasibility filtering. Using AutoDex, we collect 3,593 grasp trials across Allegro and Inspire hands on 100 diverse objects, with synchronized multi-view observations and robot-state logs. For a matched 500-trajectory collection, AutoDex requires 10.3 h versus 49.4 h for teleoperation, yielding a 4.8x throughput improvement, and grasps retrieved from the AutoDex-validated database succeed 76% versus 34% for simulation-only validation. Code and data will be publicly released.

Problem

• dexterous grasping
• real-world data collection
• grasp validation
• robotic manipulation
• automated data acquisition

Innovation

• automated data collection
• dexterous grasping
• real-world validation
• multi-view perception
• robotic reset mechanism