AdaDexGrasp: Adaptive Dexterous Grasping via 3D Visuo-Tactile Representation Fusion

2026-08-06

AI Summary

This work addresses the limitations of existing robotic grasping methods, which predominantly rely on visual input and lack adaptive mechanisms guided by tactile feedback after contact, thereby compromising robustness and generalization. To overcome this, the paper proposes a unified visuo-tactile grasping framework that associates tactile signals with individual finger identities to construct an efficient 3D joint visuo-tactile representation, tightly coupling object geometry with tactile feedback. The framework enables pre-contact grasp pose generation and feasibility prediction, as well as post-contact tactile-guided online optimization, facilitating fine-grained reasoning and dynamic adjustment of finger–object interactions. Experimental results demonstrate that the proposed method significantly improves grasping success rates in both simulation and real-world settings and exhibits strong generalization across a variety of previously unseen objects.

Abstract

Humans achieve stable and adaptive grasps by seamlessly integrating visual perception and tactile feedback, a capability that remains challenging to replicate in robotic systems. Existing robotic grasping approaches predominantly rely on visual inputs and lack mechanisms for tactile-guided adaptation after contact, limiting robustness and generalization. To address this challenge, we propose a unified visuo-tactile-fusion grasping framework that integrates grasp generation, feasibility prediction, and adaptive refinement. At its core, our method introduces an efficient visuo-tactile representation that tightly fuses object geometry with tactile feedback by associating tactile signals with finger identities. This unified representation supports contact-aware grasp pose generation during planning and tactile-guided refinement after contact, enabling the system to reason about fine-grained finger-object interactions and adjust grasps dynamically. Comprehensive experiments in both simulation and real-world environments demonstrate that our approach significantly enhances grasp success rates and generalization across diverse objects.

Problem

• dexterous grasping
• visuo-tactile fusion
• tactile feedback
• adaptive grasping
• robotic manipulation

Innovation

• visuo-tactile fusion
• adaptive grasping
• dexterous manipulation
• tactile-guided refinement
• 3D representation