Accessible Data Access and Analysis by People who are Blind or Have Low Vision

2025-06-29

AI Summary

To address information access barriers faced by blind and low-vision (BLV) individuals in data exploration and analysis, this work proposes a multimodal interactive system integrating a refreshable tactile display with a conversational agent. The system pioneers the coupling of dynamic tactile graphics with conversational natural language generation (NLG), enabling tactile–vocal co-feedback for autonomous, proactive analysis of government, health, and personal datasets. Technically, it unifies tactile rendering, text-to-speech synthesis, dialogue management, natural language understanding, and NLG modules. Prototype evaluation demonstrates that the system significantly lowers data access barriers for BLV users, improves analytical efficiency and independence, and advances paradigm innovation in accessible human–computer interaction—particularly within multimodal interfaces and intelligent assistive technologies.

Technology Category

• Humans and AI: AI for Accessibility
• Natural Language Processing: Conversational AI/Dialog Systems
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Our work aims to develop new assistive technologies that enable blind or low vision (BLV) people to explore and analyze data readily. At present, barriers exist for BLV people to explore and analyze data, restricting access to government, health and personal data, and limiting employment opportunities. This work explores the co-design and development of an innovative system to support data access, with a focus on the use of refreshable tactile displays (RTDs) and conversational agents. The envisaged system will use a combination of tactile graphics and speech to communicate with BLV users, and proactively assist with data analysis tasks. As well as addressing significant equity gaps, our work expects to produce innovations in assistive technology, multimodal interfaces, dialogue systems, and natural language understanding and generation.

Problem

• Develop assistive tech for BLV data exploration
• Overcome barriers in data access for BLV individuals
• Combine tactile graphics and speech for data analysis

Innovation

• Refreshable tactile displays for data access
• Combination of tactile graphics and speech
• Proactive conversational agents for analysis