"We do use it, but not how hearing people think": How the Deaf and Hard of Hearing Community Uses Large Language Model Tools

2024-10-28

arXiv.org

AI Summary

This study addresses the accessibility gap and usability barriers faced by Deaf and hard-of-hearing (DHH) individuals when interacting with large language models (LLMs) such as ChatGPT. Employing a mixed-methods approach—80 surveys and 11 in-depth interviews—combined with qualitative thematic analysis and quantitative statistics, it provides the first empirical investigation into DHH users’ LLM usage patterns, cultural adaptation needs, and technical constraints. Findings indicate that while LLMs hold promise for information access and communication enhancement, critical barriers persist: lack of American Sign Language (ASL) support, misrepresentation of Deaf culture, and insensitivity in interface design. The study proposes seven actionable, inclusive AI design principles, centered on two key innovations: an ASL-native integration framework and a Deaf-culture-sensitive design paradigm. These contributions advance both theoretical understanding and practical implementation of equitable, culturally responsive AI interaction.

Technology Category

• Humans and AI: AI for Accessibility
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Generative AI tools, particularly those utilizing large language models (LLMs), are increasingly used in everyday contexts. While these tools enhance productivity and accessibility, little is known about how Deaf and Hard of Hearing (DHH) individuals engage with them or the challenges they face when using them. This paper presents a mixed-method study exploring how the DHH community uses Text AI tools like ChatGPT to reduce communication barriers and enhance information access. We surveyed 80 DHH participants and conducted interviews with 11 participants. Our findings reveal important benefits, such as eased communication and bridging Deaf and hearing cultures, alongside challenges like lack of American Sign Language (ASL) support and Deaf cultural understanding. We highlight unique usage patterns, propose inclusive design recommendations, and outline future research directions to improve Text AI accessibility for the DHH community.

Problem

• Deaf and Hard of Hearing
• Large Language Model
• Accessibility Design

Innovation

• Deaf and Hard of Hearing (DHH) Community
• AI Accessibility Improvements
• Equitable Design for AI Tools