Reimagining Wearable AR Gesture Design: Physical Therapy Reasoning in Everyday Contexts

2026-03-13

AI Summary

This work addresses the limitations of current AR gesture designs, which often inherit paradigms from VR or early input devices and overlook critical factors such as user fatigue, long-term sustainability, and social readability in everyday contexts. To bridge this gap, the authors collaborate with physical therapists to develop a three-stage iterative design process grounded in ergonomic expertise. This process begins with constructing a “Daily AR Golden Ergonomic Canvas,” followed by a “Physical Therapist–Guided Gesture Translation Method,” and culminates in a “Phased Social Readability Framework.” Integrating application scenario analysis, on-device gesture elicitation, expert participatory design, and card-sorting studies, the research yields a recognizer-agnostic gesture design reference framework that supports the creation of lightweight AR gesture vocabularies aligned with ergonomic principles, sustainable interaction, and social coordination.

Technology Category

• Humans and AI: Interaction Techniques and Devices
• Computer Vision: Biometrics, Face, Gesture & Pose
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

Lightweight augmented reality (AR) glasses are increasingly entering everyday use, extending interaction design beyond short, isolated sessions. However, most existing gesture vocabularies are inherited from VR headsets or early AR goggles. These systems tend to prioritize recognizer accuracy while overlooking fatigue, sustainability, and social legibility in daily contexts. To address this gap, we collaborated with physical therapists (PTs) to reimagine gesture design for everyday AR, drawing on their expertise in safe and sustainable movement. Through a review of 104 AR applications, we identified 15 common gesture intents and implemented an on-device gesture generator. Ten licensed physical therapists, with an average of 14.8 years of professional experience, then shaped these gesture intents through three iterative stages: unaided gesture performance, PT-guided gesture substitution, and stage-aware card sorting. This work contributes (1) a PT-informed gesture translation method, (2) the Everyday-AR Golden Ergonomic Canvas, and (3) a stage-aware social legibility framework that illustrates how gesture suitability shifts with social readability. Together, these contributions provide a recognizer-agnostic reference framework for designing sustainable and socially coherent gesture vocabularies for lightweight AR glasses.

Problem

• wearable AR
• gesture design
• social legibility
• fatigue
• sustainability

Innovation

• gesture design
• physical therapy
• augmented reality
• social legibility
• ergonomic framework