Let Robots Feel Your Touch: Visuo-Tactile Cortical Alignment for Embodied Mirror Resonance

2026-05-14

AI Summary

This work addresses the absence of a computational framework enabling robots to emulate human “mirror-touch” synesthesia—the ability to vicariously experience tactile sensations by observing others—thereby hindering empathetic and socially aware interactions. The authors propose Mirror Touch Net, the first computable model that translates the neurobiological correspondence between human visual and somatosensory cortices into an embodied mirror-resonance mechanism. By aligning visual and tactile representations through semantic, distributional, and geometric constraints across multiple levels, and leveraging manifold geometry analysis together with high-resolution tactile sensing (1,140 taxels), the model achieves precise cross-modal prediction of millimeter-scale tactile signals from RGB images. Validated on robotic hands and generalizable to observed human hand interactions, the system supports a closed-loop observation–response paradigm, offering a novel pathway toward empathetic human–robot interaction.

Abstract

Observing touch on another's body can elicit corresponding tactile sensations in the observer, a phenomenon termed mirror touch that supports empathy and social perception. This visuo-tactile resonance is thought to rely on structural correspondence between visual and somatosensory cortices, yet robotic systems lack computational frameworks that instantiate this principle. Here we demonstrate that cortical correspondence can be operationalized to endow robots with mirror touch. We introduce Mirror Touch Net, which imposes semantic, distributional and geometric alignment between visual and tactile representations through multi-level constraints, enabling prediction of millimetre-scale tactile signals across 1,140 taxels on a robotic hand from RGB images. Manifold analysis reveals that these constraints reshape visual representations into geometry consistent with the tactile manifold, reducing the complexity of cross-modal mapping. Extending this alignment framework to cross-domain observations of human hands enables tactile prediction and reflexive responses to observed human touch. Our results link a neural principle of visuo-tactile resonance to robotic perception, providing an explainable route towards anticipatory touch and empathic human-robot interaction. Code is available at https://github.com/fun0515/Mirror-Touch-Net.

Problem

• mirror touch
• visuo-tactile resonance
• embodied perception
• human-robot interaction
• cortical alignment

Innovation

• mirror touch
• visuo-tactile alignment
• cortical correspondence
• tactile prediction
• embodied resonance