An Interpretable Deep Learning Framework for Material Perception and Classification from Multisensory Tactile Data

2026-08-22

AI Summary

研究通过开发一个包含三个深度学习模型的框架，解决了从多感官触觉数据到材料感知和分类的问题，该框架不依赖手工特征，并结合集成梯度方法提高了可解释性。

Abstract

Human tactile perception relies on complex multisensory cues. Yet the relationship between tactile signals and perceptual representations remains poorly understood, limiting the integration of touch in digital environments and human-like robotic perception. To address this gap, we developed a computational framework comprising three interconnected deep learning models that map multisensory touch data to material perception, without relying on hand-crafted features. The models represent progressively different routes from tactile signals to material class: from low-level interaction signals to perceptual attribute distributions (Model 1), from predicted attribute distributions to material classification (Model 2), and directly from tactile signals to material categories, bypassing intermediate representations (Model 3). By combining deep learning with Integrated Gradients, the framework achieved high accuracy while offering interpretability, revealing which sensory modalities most strongly drive its decisions. Our results show that deep learning can approach near-perfect material classification when unconstrained by intermediate perceptual stages, but matching human-like performance is harder once those stages are modeled explicitly. Notably, thermal cues emerged as particularly informative across all models, providing robust signals for material differentiation. The results offer a computational account of how tactile signals lead to material perception and show how interpretable deep learning can both approach human-level performance and reveal cues that robotic and haptic systems need to incorporate.

Problem

• tactile perception
• multisensory cues
• material classification
• deep learning
• interpretability

Innovation

• Interpretable Deep Learning
• Multisensory Tactile Data
• Material Classification
• Integrated Gradients
• Thermal Cues