Explore, Listen, Inspect: Supporting Multimodal Interaction with 3D Surface and Point Data Visualizations

2025-08-11

AI Summary

Current web-based 3D surface and point cloud visualization tools rely heavily on visual interaction, rendering them inaccessible to blind and low-vision (BLV) users in browser environments. To address this, we propose DIXTRAL—the first browser-native, synchronous multimodal 3D data visualization system designed specifically for BLV users. It integrates data sonification, dynamic textual descriptions, and optional visual feedback, and supports keyboard and game controller input. Its interaction logic was co-designed with BLV stakeholders and refined through iterative user studies. Experimental evaluation demonstrates that DIXTRAL significantly improves BLV users’ ability to recognize structural patterns in 3D scalar fields, perform spatial orientation, and conduct efficient exploratory analysis. This work contributes a reusable architectural paradigm and empirically grounded design guidelines for inclusive scientific visualization.

Technology Category

• Computer Vision: 3D Computer Vision
• Humans and AI: AI for Accessibility
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Web Mining and Content Analysis: Web data visualization
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Security and Privacy: Data transparency and provenance

Abstract

Blind and low-vision (BLV) users remain largely excluded from three-dimensional (3D) surface and point data visualizations due to the reliance on visual interaction. Existing approaches inadequately support non-visual access, especially in browser-based environments. This study introduces DIXTRAL, a hosted web-native system, co-designed with BLV researchers to address these gaps through multimodal interaction. Conducted with two blind and one sighted researcher, this study took place over sustained design sessions. Data were gathered through iterative testing of the prototype, collecting feedback on spatial navigation, sonification, and usability. Co-design observations demonstrate that synchronized auditory, visual, and textual feedback, combined with keyboard and gamepad navigation, enhances both structure discovery and orientation. DIXTRAL aims to improve access to 3D continuous scalar fields for BLV users and inform best practices for creating inclusive 3D visualizations.

Problem

• Enabling BLV users to access 3D surface and point visualizations
• Addressing non-visual interaction gaps in browser-based environments
• Improving multimodal navigation for 3D data exploration

Innovation

• Web-native system with multimodal interaction
• Synchronized auditory, visual, textual feedback
• Keyboard and gamepad navigation for BLV users