Learning to Feel Materials from Multisensory Tactile Data via Interpretable Models

2026-05-28

AI Summary

Human tactile perception relies on multimodal cues, yet the mapping between low-level tactile signals and high-level perceptual representations remains poorly understood, hindering the application of tactile technologies in digital and robotic systems. This work proposes an interpretable computational framework that, for the first time, systematically integrates thermal conduction and deformation (compliance) cues across pressing, static contact, and sliding interactions. The framework comprises three interconnected models that map physical interaction features to psychophysical perceptual attributes and enable material classification. By combining multimodal tactile sensing, psychophysical modeling, and interpretable machine learning, the approach significantly improves material identification accuracy and reveals the critical roles of thermal and compliance cues in perceptual modeling and multimodal cue integration.

Abstract

Human tactile perception of materials relies on complex multisensory touch cues, yet the relationship between low-level tactile signals and perceptual representations remains poorly understood. This knowledge gap hinders the integration of touch in digital environments and the development of robots capable of human-like tactile perception. Here, we present an interpretable computational framework for modeling human material perception and recognition using multisensory touch data. Our framework comprises three interconnected models: Model 1 maps finger-surface interaction features to psychophysical sensory attributes, Model 2 classifies materials based on these perceptual representations, and Model 3 directly classifies materials from tactile features. The results showed that combining information from pressing, static contact, and sliding interactions improves prediction accuracy, and that thermal cues are particularly informative for both perceptual modeling and material classification. These findings highlight the importance of thermal and compliance cues, which remain underrepresented in current robotic fingers and haptic displays. Incorporating such cues may enhance artificial systems' ability to approximate human material perception and guide the design of more perceptually grounded haptic interfaces.

Problem

• tactile perception
• multisensory touch
• material recognition
• haptic interfaces
• psychophysical attributes

Innovation

• interpretable models
• multisensory tactile data
• material perception
• thermal cues
• haptic interfaces