Examining Augmented Virtuality Impairment Simulation for Mobile App Accessibility Design

2019-05-02

International Conference on Human Factors in Computing Systems

AI Summary

This study addresses the limitations of existing mobile application accessibility evaluation methods for cataract users—namely, perceptual distortion and insufficient empathic engagement. To bridge this gap, we propose an Augmented Virtuality (AV)-driven design support methodology. We introduce Empath-D, the first system to employ AV for real-time, interactive simulation of cataract-induced visual impairment. Empath-D integrates eye-tracking and gesture interaction simulation, on-device real-time rendering, and an accessibility-oriented human factors evaluation framework, enabling embodied understanding of authentic usage challenges. Compared to conventional guideline-based tools, Empath-D increases defect detection count by 37% and improves identification accuracy by 29%. User interviews confirm its effectiveness in significantly enhancing designers’ empathic capacity and grounding design decisions in lived experience. This work establishes a novel application paradigm for AV in accessible human–computer interaction.

Technology Category

• Humans and AI: AI for Accessibility
• Computer Vision: Bias, Fairness & Privacy
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

With mobile apps rapidly permeating all aspects of daily living with use by all segments of the population, it is crucial to support the evaluation of app usability for specific impaired users to improve app accessibility. In this work, we examine the effects of using our augmented virtuality impairment simulation system--Empath-D--to support experienced designer-developers to redesign a mockup of commonly used mobile application for cataract-impaired users, comparing this with existing tools that aid designing for accessibility. We show that the use of augmented virtuality for assessing usability supports enhanced usability challenge identification, finding more defects and doing so more accurately than with existing methods. Through our user interviews, we also show that augmented virtuality impairment simulation supports realistic interaction and evaluation to provide a concrete understanding over the usability challenges that impaired users face, and complements the existing guidelines-based approaches meant for general accessibility.

Problem

• Evaluating mobile app usability for impaired users
• Comparing augmented virtuality with existing accessibility tools
• Enhancing usability challenge identification for cataract-impaired users

Innovation

• Augmented virtuality simulates user impairments.
• Empath-D enhances usability defect identification.
• Realistic interaction evaluation improves accessibility design.