Learning Object Compliance via Young's Modulus from Single Grasps using Camera-Based Tactile Sensors

2024-06-18

AI Summary

To address the challenge of estimating Young’s modulus (E) across diverse object shapes and materials during a single robotic grasp, this paper introduces the first hybrid framework integrating analytical mechanical modeling with a multi-tower neural network. The method leverages contact image sequences from camera-based tactile sensors, combined with proprioceptive and force measurements, to construct a geometry-agnostic and material-robust compliance estimation model. An analytical module encodes Hooke’s law and contact geometry priors, while a data-driven module learns the tactile deformation-to-material mapping. Evaluated on 285 real-world objects, the framework achieves 74.2% accuracy in Young’s modulus estimation (error ≤1 order of magnitude), substantially outperforming purely analytical (28.9%) and purely data-driven baselines (65.0%). This work marks the first demonstration of generalizable, high-accuracy elastic parameter identification under single parallel grasps.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Compliance is a useful parametrization of tactile information that humans often utilize in manipulation tasks. It can be used to inform low-level contact-rich actions or characterize objects at a high-level. In robotic manipulation, existing approaches to estimate compliance have struggled to generalize across both object shape and material. Using camera-based tactile sensors, proprioception, and force measurements, we present a novel approach to estimate object compliance as Young's modulus (E) from parallel grasps. We evaluate our method over a novel dataset of 285 common objects, including a wide array of shapes and materials with Young's moduli ranging from 5.0 kPa to 250 GPa. Combining analytical and data-driven approaches, we develop a hybrid system using a multi-tower neural network to analyze a sequence of tactile images from grasping. This system is shown to estimate the Young's modulus of unseen objects within an order of magnitude at 74.2% accuracy across our dataset. This is an improvement over purely analytical and data-driven baselines which exhibit 28.9% and 65.0% accuracy respectively. Importantly, this estimation system performs irrespective of object geometry and demonstrates increased robustness across material types.

Problem

• Estimating object compliance via Young's modulus from single grasps.
• Generalizing compliance estimation across diverse shapes and materials.
• Improving accuracy and robustness in robotic manipulation tasks.

Innovation

• Camera-based tactile sensors estimate Young's modulus.
• Hybrid system combines analytical and data-driven approaches.
• Multi-tower neural network analyzes tactile image sequences.