GraspSense: Physically Grounded Grasp and Grip Planning for a Dexterous Robotic Hand via Language-Guided Perception and Force Maps

2026-04-07

AI Summary

This work addresses the common oversight in existing grasping methods of spatially non-uniform mechanical properties on object surfaces, which often leads to damage when contacts occur at fragile regions. The authors propose a novel grasping framework that integrates language instructions, 3D reconstruction, and physical awareness: leveraging SAM3D for language-guided 3D reconstruction, they perform physics-informed geometric analysis to generate local contact force tolerance maps, which are then used to filter and re-rank candidate grasp poses according to task consistency and force-map awareness. During execution, an adaptive impedance controller dynamically modulates finger stiffness based on contact points. This approach is the first to incorporate local mechanical tolerance maps throughout the entire grasping pipeline, reframing dexterous manipulation from a purely geometric problem into a joint optimization under physical constraints. Experiments demonstrate stable selection of high-strength contact regions and maintenance of grasp forces within safe thresholds on paper, plastic, and glass cups.

Abstract

Dexterous robotic manipulation requires more than geometrically valid grasps: it demands physically grounded contact strategies that account for the spatially non-uniform mechanical properties of the object. However, existing grasp planners typically treat the surface as structurally homogeneous, even though contact in a weak region can damage the object despite a geometrically perfect grasp. We present a pipeline for grasp selection and force regulation in a five-fingered robotic hand, based on a map of locally admissible contact loads. From an operator command, the system identifies the target object, reconstructs its 3D geometry using SAM3D, and imports the model into Isaac Sim. A physics-informed geometric analysis then computes a force map that encodes the maximum lateral contact force admissible at each surface location without deformation. Grasp candidates are filtered by geometric validity and task-goal consistency. When multiple candidates are comparable under classical metrics, they are re-ranked using a force-map-aware criterion that favors grasps with contacts in mechanically admissible regions. An impedance controller scales the stiffness of each finger according to the locally admissible force at the contact point, enabling safe and reliable grasp execution. Validation on paper, plastic, and glass cups shows that the proposed approach consistently selects structurally stronger contact regions and keeps grip forces within safe bounds. In this way, the work reframes dexterous manipulation from a purely geometric problem into a physically grounded joint planning problem of grasp selection and grip execution for future humanoid systems.

Problem

• dexterous manipulation
• grasp planning
• force map
• contact mechanics
• physically grounded

Innovation

• force map
• physically grounded grasp planning
• dexterous manipulation
• impedance control
• language-guided perception