Motion Design Principles for Accessible Video-based Learning: Addressing Cognitive Challenges for Deaf and Hard of Hearing Learners

2024-09-30

arXiv.org

AI Summary

Deaf and hard-of-hearing (DHH) learners experience cognitive overload in video-based learning due to conflicting audiovisual information, while conventional captioning approaches neglect the cognitive load associated with concurrent visual–textual processing. To address this, we propose five Motion Design Principles specifically for DHH learners, replacing static captions with dynamic visual cues—such as motion trajectories, rhythmic synchronization, and spatial mapping—to enhance visual attention guidance and spatiotemporal audiovisual coherence. Through a two-phase empirical study—including task analysis and a controlled experiment with 16 DHH participants—we evaluated cognitive load (using validated cognitive load scales) and learning satisfaction. Results demonstrate significant reductions in extraneous cognitive load and temporal pressure, alongside improved comprehension efficiency and learner satisfaction. This work transcends the caption-centric paradigm by establishing, for the first time, a systematic, motion-grammar–based framework for accessible video design, offering both theoretical foundations and practical guidelines for inclusive educational technology.

Technology Category

• Computer Vision: Motion & Tracking
• Humans and AI: AI for Accessibility
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Deaf and Hard-of-Hearing (DHH) learners face unique challenges in video-based learning due to the complex interplay between visual and auditory information in videos. Traditional approaches to making video content accessible primarily focus on captioning, but these solutions often neglect the cognitive demands of processing both visual and textual information simultaneously. This paper introduces a set of extit{Motion} design guidelines, aimed at mitigating these cognitive challenges and improving video learning experiences for DHH learners. Through a two-phase research, we identified five key challenges, including misaligned content and visual overload. We proposed five design principles accordingly. User study with 16 DHH participants showed that improving visual-audio relevance and guiding visual attention significantly enhances the learning experience by reducing physical demand, alleviating temporal pressure, and improving learning satisfaction. Our findings highlight the potential of Motion design to transform educational content for DHH learners, and we discuss implications for inclusive video learning tools.

Problem

• Addressing cognitive challenges in video learning for DHH learners
• Improving visual-audio relevance to reduce cognitive load
• Designing motion principles to enhance DHH learning experience

Innovation

• Motion design guidelines for DHH learners
• Improving visual-audio relevance in videos
• Guiding visual attention to reduce cognitive load