A High-Force Gripper with Embedded Multimodal Sensing for Powerful and Perception Driven Grasping

2024-11-22

IEEE-RAS International Conference on Humanoid Robots

AI Summary

Current humanoid robot end-effectors suffer from limited payload capacity—typically far below that of the host manipulator—and lack integrated perception, resulting in poor grasping robustness and susceptibility to occlusion by arm motion. To address these limitations, we propose a modular, high-force, multimodal robotic hand. It incorporates high-torque motors enabling a sustained grasp force of 110 N; embeds on-hand RGB and time-of-flight (ToF) depth sensors, a six-axis IMU, and an omnidirectional microphone to establish a real-time, edge-based multimodal perception loop; and introduces, for the first time, a unified hardware architecture integrating high-force actuation with embedded sensing. Furthermore, we propose a novel dynamic load assessment metric jointly informed by manipulator dynamics and gripper thermal state. Experimental results demonstrate over 92% grasping success rate under dynamic occlusion and a threefold increase in thermally stable operational duration.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Modern humanoid robots have shown their promising potential for executing various tasks involving the grasping and manipulation of objects using their end-effectors. Nevertheless, in the most of the cases, the grasping and manipulation actions involve low to moderate payload and interaction forces. This is due to limitations often presented by the end-effectors, which can not match their arm-reachable payload, and hence limit the payload that can be grasped and manipulated. In addition, grippers usually do not embed adequate perception in their hardware, and grasping actions are mainly driven by perception sensors installed in the rest of the robot body, frequently affected by occlusions due to the arm motions during the execution of the grasping and manipulation tasks. To address the above, we developed a modular high grasping force gripper equipped with embedded multi-modal perception functionalities. The proposed gripper can generate a grasping force of 110 N in a compact implementation. The high grasping force capability is combined with embedded multi-modal sensing, which includes an eye-in-hand camera, a Time-of-Flight (ToF) distance sensor, an Inertial Measurement Unit (IMU) and an omnidirectional microphone, permitting the implementation of perception-driven grasping functionalities. We extensively evaluated the grasping force capacity of the gripper by introducing novel payload evaluation metrics that are a function of the robot arm’s dynamic motion and gripper thermal states. We also evaluated the embedded multi-modal sensing by performing perception-guided enhanced grasping operations.

Problem

• Develops high-force gripper for heavy payload grasping
• Integrates multimodal sensing to overcome occlusion limitations
• Enables perception-driven grasping with embedded sensors

Innovation

• Modular high-force gripper with 110 N capacity
• Embedded multi-modal sensing for perception-driven grasping
• Novel payload metrics based on motion and thermal states