Arm-Aware Guided Dexterous Grasp Generation with Arm-Agnostic Grasp Models

2026-08-17

AI Summary

This study addresses the deployment challenges arising from existing dexterous grasp generation methods that neglect robotic arm constraints. We propose an arm-aware grasping framework that formulates arm constraints as a differentiable optimization problem, enabling gradient-guided diffusion models to jointly optimize hand poses and arm configurations. Theoretically proven equivalent to controlled sampling, this approach achieves adaptive generation without retraining. Extensive experiments across six scene categories and ten thousand objects demonstrate that our method significantly improves feasible grasp success rates in highly constrained environments. By effectively resolving collision avoidance and limited workspace issues, this work validates its practical advantages for real-world robotic manipulation tasks where kinematic feasibility is critical.

Abstract

Dexterous grasp generation that considers arm-related constraints is crucial in real-world scenarios involving arm environment collision avoidance, workspace boundary grasps, and consecutive grasping. Existing hand-centric grasp models, which primarily focus on the floating hand's pose, are insufficient for such cases. Conventional arm-aware methods either rely on rejection sampling to discard infeasible samples or require retraining on arm-specific data, leading to low sample efficiency under adverse conditions or limited generalization across different robots and environments. To overcome these limitations, this letter presents an arm-aware dexterous grasp generation framework that leverages pretrained arm-agnostic grasp models while integrating arm and environmental information only at inference time. Specifically, we formulate arm-aware constrained grasp generation as a joint optimization of hand pose and arm configuration, and derive closed-form gradients for arm-related constraints. Assuming the hand pose distribution is represented by a diffusion model, we prove that gradient-based optimization is equivalent to guided diffusion sampling, steering near-feasible samples toward the feasible region. Through comprehensive evaluation involving 10k objects across 6 scenarios, we demonstrate that the proposed framework generates feasible grasps in highly constrained settings with significantly higher probability, highlighting its advantages in real-world applications. Supplementary materials and appendix are available at https://arm-aware-dexgrasp.github.io/.

Problem

• Dexterous Grasp Generation
• Arm-Aware Constraints
• Collision Avoidance
• Workspace Boundary
• Generalization

Innovation

• Arm-Aware Guided Diffusion
• Closed-Form Gradients
• Joint Optimization
• Arm-Agnostic Models
• Inference-Time Adaptation