Pinching Visuo-haptic Display: Investigating Cross-Modal Effects of Visual Textures on Electrostatic Cloth Tactile Sensations

2025-10-12

International Conference on Multimodal Interaction

AI Summary

This study investigates how visual texture modulates cross-modal tactile perception in electrostatic fabric haptic displays. We employed a conductive-fabric-based electrostatic adhesion haptic device synchronized with a spatially registered virtual reality fabric rendering system to deliver congruent or incongruent visual textures alongside tactile roughness cues. Results demonstrate that visual texture significantly modulates subjective perceptions of stiffness (voile), warmth (toweling), and roughness (denim), confirming that visual augmentation effectively extends the material expressiveness of single-dimension haptic devices—those capable only of modulating surface friction. In contrast, perceived thickness remained unaffected, as pinch-based interactions provided dominant haptic cues. This work presents the first systematic validation of cross-modal visual–tactile integration for dimensional expansion in fabric interaction, establishing both theoretical foundations and practical implementation strategies for multimodal textile interfaces.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Humans and AI: Interaction Techniques and Devices
• Computer Vision: Multi-modal Vision

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Economics, Online Markets and Human Computation: The sharing economy
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health

Abstract

Tactile representations of fabrics are critical in virtual retail and VR fitting applications. A pinching tactile display (PTD) using electrostatic adhesion is a fabric-type display that can flexibly adjust tactile sensations by adjusting voltage and frequency, and is expected to be applied to the remote transmission of fabric tactile sensations. However, no significant differences in tactile sensation other than roughness were observed. Building on established research in cross-modal perception, we investigated whether overlaying visual fabric textures could alter the tactile expressiveness of PTDs. We developed a system that integrated a systematically controlled conductive fabric-based PTD with spatially registered VR fabric simulations. Participants rated four tactile properties (roughness, stiffness, thickness, warmth) while touching the same conductive substrate under different visual texture conditions (denim, gauze, toweling, voile). The results showed that voile texture significantly affected stiffness (p=.00887); toweling texture tended to affect warmth (p=.0658); and denim texture tended to affect roughness (p=.0972). These results suggest that virtual texture overlay can affect the warmth, stiffness, and roughness of the PTD. Since the PTD can only change roughness, these results suggest that texture overlay enables the display of tactile sensations that the PTD alone cannot express. On the other hand, no significant influence of texture on thickness was observed. This suggests the pinching motion provided a highly reliable tactile cue, which may have led to visual information being underrated and thus not reaching a significant level. These findings offer strategic insights for employing visual textures to enhance perceived fabric properties, paving the way for deployment in virtual fashion retail and VR fitting applications.

Problem

• Investigating visual texture effects on electrostatic tactile perception
• Exploring cross-modal influence between visual roughness and tactile friction
• Developing visuo-haptic system for realistic virtual fabric interaction

Innovation

• Visual textures modulate electrostatic tactile sensations
• Visuo-haptic system simulates pinching virtual fabrics
• Visual roughness amplifies perceived friction in haptics