TensorTouch: Calibration of Tactile Sensors for High Resolution Stress Tensor and Deformation for Dexterous Manipulation

2025-06-09

AI Summary

Existing optical tactile sensors struggle with multi-point, non-coplanar, and large-deformation contact sensing in dexterous manipulation, as their raw images lack physical interpretability and cross-platform generalizability. Method: We propose the first calibration framework integrating finite element analysis (FEA) with physics-informed deep learning to enable interpretable, pixel-wise inversion of stress tensors, deformation fields, and contact force distributions. Our approach features stress–deformation coupling decoupling, multimodal image regression, and cross-sensor calibration. Contribution/Results: Experiments demonstrate sub-millimeter contact localization accuracy, contact force estimation error <8%, and 90% success rate in a dual-rope selective grasping task—significantly advancing perception–actuation integration under high-density contact conditions.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Low Level & Physics-based Vision
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Advanced dexterous manipulation involving multiple simultaneous contacts across different surfaces, like pinching coins from ground or manipulating intertwined objects, remains challenging for robotic systems. Such tasks exceed the capabilities of vision and proprioception alone, requiring high-resolution tactile sensing with calibrated physical metrics. Raw optical tactile sensor images, while information-rich, lack interpretability and cross-sensor transferability, limiting their real-world utility. TensorTouch addresses this challenge by integrating finite element analysis with deep learning to extract comprehensive contact information from optical tactile sensors, including stress tensors, deformation fields, and force distributions at pixel-level resolution. The TensorTouch framework achieves sub-millimeter position accuracy and precise force estimation while supporting large sensor deformations crucial for manipulating soft objects. Experimental validation demonstrates 90% success in selectively grasping one of two strings based on detected motion, enabling new contact-rich manipulation capabilities previously inaccessible to robotic systems.

Problem

• Calibrating tactile sensors for high-resolution stress tensor measurement
• Enabling dexterous manipulation with multiple simultaneous contacts
• Converting raw optical tactile data into interpretable physical metrics

Innovation

• Integrates finite element analysis with deep learning
• Extracts pixel-level stress tensors and deformation
• Achieves sub-millimeter accuracy in force estimation