Grasp to Act: Dexterous Grasping for Tool Use in Dynamic Settings

2026-02-24

AI Summary

This work addresses the limitations of existing dexterous grasping methods, which predominantly rely on static geometric stability and struggle to maintain robustness under dynamic external forces—such as impacts or torques—during tool manipulation. To overcome this, the authors propose Grasp-to-Act, a hybrid system that uniquely integrates human demonstration-guided robust grasp configurations with a residual adaptive controller. By combining physics-based simulation optimization, reinforcement learning, and demonstration-driven grasp synthesis, the approach enables real-time joint-level adjustments during functional tasks to suppress both translational and rotational slip while accurately tracking desired trajectories. Notably, the method achieves zero-shot sim-to-real transfer without task-specific fine-tuning, demonstrating superior performance across five dynamic tasks—hammering, sawing, cutting, stirring, and scooping—with significantly reduced slip and higher task success rates, validated on a 16-degree-of-freedom dexterous hand.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Achieving robust grasping with dexterous hands remains challenging, especially when manipulation involves dynamic forces such as impacts, torques, and continuous resistance--situations common in real-world tool use. Existing methods largely optimize grasps for static geometric stability and often fail once external forces arise during manipulation. We present Grasp-to-Act, a hybrid system that combines physics-based grasp optimization with reinforcement-learning-based grasp adaptation to maintain stable grasps throughout functional manipulation tasks. Our method synthesizes robust grasp configurations informed by human demonstrations and employs an adaptive controller that residually issues joint corrections to prevent in-hand slip while tracking the object trajectory. Grasp-to-Act enables robust zero-shot sim-to-real transfer across five dynamic tool-use tasks--hammering, sawing, cutting, stirring, and scooping--consistently outperforming baselines. Across simulation and real-world hardware trials with a 16-DoF dexterous hand, our method reduces translational and rotational in-hand slip and achieves the highest task completion rates, demonstrating stable functional grasps under dynamic, contact-rich conditions.

Problem

• dexterous grasping
• dynamic manipulation
• tool use
• in-hand slip
• grasp stability

Innovation

• dexterous grasping
• dynamic tool use
• grasp adaptation
• sim-to-real transfer
• reinforcement learning