Human Universal Grasping

2026-06-15

AI Summary

Existing approaches struggle to achieve general and naturalistic grasping of everyday objects with multi-fingered robotic hands. This work proposes a flow-matching-based generative model that leverages the large-scale first-person human grasping dataset 1M-HUGs—comprising one million frames—to predict diverse, anthropomorphic grasps from a single RGB-D image. By fusing RGB and depth information and integrating the MANO hand representation with grasp retargeting techniques, the method enables zero-shot transfer to various robotic hand morphologies. Evaluated on the standardized HUG-Bench benchmark with 90 unseen objects and an additional set of 30 real-world challenging objects, the approach improves grasp success rates by 23%–34% over state-of-the-art methods and demonstrates effective deployment across multiple home environments.

Abstract

Humans can grasp objects effortlessly, whereas multi-fingered robots are far from this level of generality. We argue that the most natural source of robot grasping data is from humans, who pick up thousands of objects every day. We present HUG, a flow-matching model that generates diverse human grasps for any user-specified object in a single RGB-D image captured from a stereo camera. Using smart glasses, we first collect 1M-HUGs, an egocentric dataset of human grasps spanning 1M frames (27.8 hrs) and 6,707 object instances across 41 buildings. Next, to model the distribution of natural human grasps, our novel flow-matching model fuses RGB and depth observations to output a grasp parameterized by wrist translation, wrist rotation, and MANO hand pose. Predicted grasps can be retargeted to various robot hands, enabling zero-shot grasping in everyday scenes. To standardize evaluation, we build a new simulated benchmark, HUG-Bench, of 90 unseen objects from five geometric categories and various sizes, with metric-scale 3D meshes. We evaluate HUG in the real world on the 30-object test set of HUG-Bench across multiple stereo cameras, robot embodiments, and household environments. HUG outperforms the state-of-the-art grasping baselines by +23% and +34% on our challenging object set. Code, data, benchmark, checkpoints, and an interactive demo are released on our website: https://grasping.io/

Problem

• robot grasping
• human grasping
• RGB-D
• multi-fingered robots
• grasp generation

Innovation

• flow-matching
• human grasping
• RGB-D
• zero-shot grasping
• egocentric dataset