🤖 AI Summary
Robotic manipulation of thin, deformable materials (e.g., fabric, paper) often fails due to inaccurate layer counting during grasping. To address this, we propose a multimodal tactile sensing method for layer estimation, leveraging dynamic friction-based motions and DenseTact 2.0 optical tactile sensing. Our approach integrates human-inspired friction gestures with spatiotemporal tactile features—including optical flow, six-axis force-torque, and joint-state signals—processed via a Transformer architecture to model sequential tactile dynamics. We introduce the first open-source, multimodal dataset for thin-material layer annotation, enabling systematic benchmarking. Experiments demonstrate 98.21% classification accuracy for fabric layers and 81.25% for paper layers, confirming the critical role of dynamic tactile cues and temporal modeling in robust layer discrimination. This work establishes a generalizable perception paradigm for reliable robotic handling of thin, layered materials.
📝 Abstract
Manipulation of thin materials is critical for many everyday tasks and remains a significant challenge for robots. While existing research has made strides in tasks like material smoothing and folding, many studies struggle with common failure modes (crumpled corners/edges, incorrect grasp con-figurations) that a preliminary step of layer detection can solve. We present a novel method for classifying the number of grasped material layers using a custom gripper equipped with DenseTact 2.0 optical tactile sensors. After grasping a thin material, the gripper performs an anthropomorphic rubbing motion while collecting optical flow, 6-axis wrench, and joint state data. Using this data in a transformer-based network achieves a test accuracy of 98.21% in correctly classifying the number of grasped cloth layers, and 81.25% accuracy in classifying layers of grasped paper, showing the effectiveness of our dynamic rubbing method. Evaluating different inputs and model architectures highlights the usefulness of tactile sensor information and a transformer model for this task. A comprehensive dataset of 568 labeled trials (368 for cloth and 200 for paper) was collected and made open-source along with this paper. Our project page is available at https://armlabstanford.github.io/dynamic-cloth-detection.