"It Brought the Model to Life": Exploring the Embodiment of Multimodal I3Ms for People who are Blind or have Low Vision

2025-02-20

AI Summary

This study addresses the limited sense of engagement and trust experienced by blind and low-vision (BLV) users when interacting with interactive 3D-printed models (I3Ms). We propose an embodied multimodal interaction design integrating tactile gesture recognition, anthropomorphic speech synthesis, and rhythm-based vibrotactile feedback. To our knowledge, this is the first work to introduce anthropomorphic voice and embodied vibration modeling into I3Ms for BLV users, enabling a programmable, user-centered, multi-channel feedback system. In a controlled experiment with 12 BLV participants, the embodied I3Ms significantly improved perceived vividness (*p* < 0.01) and interaction engagement (*p* < 0.05); however, effects on trust were context-dependent—significantly enhanced under high task–model alignment, but not significant under low alignment. This work establishes a novel paradigm for accessible embodied interaction and provides empirical evidence supporting human-centered design of tangible interfaces for BLV populations.

Technology Category

• Humans and AI: Interaction Techniques and Devices
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

3D-printed models are increasingly used to provide people who are blind or have low vision (BLV) with access to maps, educational materials, and museum exhibits. Recent research has explored interactive 3D-printed models (I3Ms) that integrate touch gestures, conversational dialogue, and haptic vibratory feedback to create more engaging interfaces. Prior research with sighted people has found that imbuing machines with human-like behaviours, i.e., embodying them, can make them appear more lifelike, increasing social perception and presence. Such embodiment can increase engagement and trust. This work presents the first exploration into the design of embodied I3Ms and their impact on BLV engagement and trust. In a controlled study with 12 BLV participants, we found that I3Ms using specific embodiment design factors, such as haptic vibratory and embodied personified voices, led to an increased sense of liveliness and embodiment, as well as engagement, but had mixed impact on trust.

Problem

• Design embodied I3Ms for BLV engagement
• Explore haptic feedback in I3Ms
• Assess trust impact of embodied I3Ms

Innovation

• Interactive 3D-printed models
• Haptic vibratory feedback
• Embodied personified voices