Grasping a Handful: Sequential Multi-Object Dexterous Grasp Generation

2025-03-28

AI Summary

Multi-object dexterous grasping faces challenges from limited hand degrees of freedom (DoFs) and difficulty in modeling sequential inter-object interactions. Method: This paper proposes SeqGrasp, a stepwise multi-object grasping generation framework, and SeqDiffuser, an associated diffusion model. We introduce the first sequential grasping synthesis paradigm, integrating partial-DoF modeling, kinematic constraint embedding, and diffusion-based probabilistic modeling. Additionally, we construct SeqDataset—the first large-scale, sequential grasping dataset for the Allegro Hand. Contribution/Results: Experiments on both simulation and real-robot platforms demonstrate that our approach improves grasping success rates by 8.71%–43.33% over the state-of-the-art MultiGrasp. SeqDiffuser achieves generation speeds approximately 1000× faster than SeqGrasp and MultiGrasp, while maintaining high stability, strong generalization across unseen objects and configurations, and real-time feasibility.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: Manipulation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We introduce the sequential multi-object robotic grasp sampling algorithm SeqGrasp that can robustly synthesize stable grasps on diverse objects using the robotic hand's partial Degrees of Freedom (DoF). We use SeqGrasp to construct the large-scale Allegro Hand sequential grasping dataset SeqDataset and use it for training the diffusion-based sequential grasp generator SeqDiffuser. We experimentally evaluate SeqGrasp and SeqDiffuser against the state-of-the-art non-sequential multi-object grasp generation method MultiGrasp in simulation and on a real robot. The experimental results demonstrate that SeqGrasp and SeqDiffuser reach an 8.71%-43.33% higher grasp success rate than MultiGrasp. Furthermore, SeqDiffuser is approximately 1000 times faster at generating grasps than SeqGrasp and MultiGrasp.

Problem

• Generates stable sequential grasps for multiple objects
• Improves grasp success rates over existing methods
• Speeds up grasp generation significantly

Innovation

• SeqGrasp algorithm for multi-object grasping
• Diffusion-based SeqDiffuser for fast grasp generation
• Partial DoF usage in robotic hand control