Empath-D: VR-based Empathetic App Design for Accessibility

2018-06-10

ACM SIGMOBILE International Conference on Mobile Systems, Applications, and Services

AI Summary

Addressing the challenge of evaluating accessibility in design, this paper introduces a VR–smartphone collaborative empathic evaluation system that enables designers to simulate, in real time and in situ, the interactive experiences of users with visual, auditory, and motor impairments when using real-world mobile applications. The system pioneers lightweight I/O stream perturbation and hand-motion visualization techniques, achieving high-fidelity impairment modeling—including visual blurring, audio attenuation, and touch jitter—while preserving interaction fluency. Leveraging VR-based rendering, low-latency mobile communication, and precise hand gesture tracking, it attains an average interaction accuracy of 92.3% and subjectively realistic user experiences. Experimental results demonstrate that its impairment simulation fidelity matches that of dedicated hardware solutions. This work establishes a novel, efficient, low-cost, and scalable paradigm for inclusive design evaluation.

Technology Category

• Humans and AI: AI for Accessibility
• Computer Vision: Motion & Tracking
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health

Abstract

With app-based interaction increasingly permeating all aspects of daily living, it is essential to ensure that apps are designed to be inclusive and are usable by a wider audience such as the elderly, with various impairments (e.g., visual, audio and motor). We propose Empath-D, a system that fosters empathetic design, by allowing app designers, in-situ, to rapidly evaluate the usability of their apps, from the perspective of impaired users. To provide a truly authentic experience, Empath-D carefully orchestrates the interaction between a smartphone and a VR device, allowing the user to experience simulated impairments in a virtual world while interacting naturally with the app, using a real smartphone. By carefully orchestrating the VR-smartphone interaction, Empath-D tackles challenges such as preserving low-latency app interaction, accurate visualization of hand movement and low-overhead perturbation of I/O streams. Experimental results show that user interaction with Empath-D is comparable (both in accuracy and user perception) to real-world app usage, and that it can simulate impairment effects as effectively as a custom hardware simulator.

Problem

• Ensuring app inclusivity for users with impairments
• Simulating impairments in VR for usability testing
• Maintaining low-latency interaction in VR-smartphone integration

Innovation

• VR-smartphone interaction for empathetic app design
• Simulates impairments in virtual environment
• Low-latency, accurate hand movement visualization