ExplorAR: Assisting Older Adults to Learn Smartphone Apps through AR-powered Trial-and-Error with Interactive Guidance

2025-08-02

AI Summary

To address the difficulty older adults face in learning smartphone applications via conventional video tutorials—due to age-related declines in cognitive and motor abilities—this study proposes an augmented reality (AR)-based trial-and-error learning support method. The approach overlays contextualized, interactive visual guidance onto real-world interaction environments in real time, enabling users to autonomously explore, immediately detect, and self-correct operational errors—thereby enhancing cognitive engagement and self-efficacy. Its key innovation lies in the first deep integration of AR into a closed-loop trial-and-error learning framework, coupled with a behavior-feedback-driven dynamic guidance mechanism. User experiments demonstrate that, compared to video tutorials, the method significantly improves learning efficiency (+32%), cognitive engagement (p < 0.01), and operational confidence (+41%). This work offers a scalable technical pathway toward digital inclusion for older adults.

Technology Category

• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Intelligent Robots: Behavior Learning & Control
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Older adults tend to encounter challenges when learning to use new smartphone apps due to age-related cognitive and physical changes. Compared to traditional support methods such as video tutorials, trial-and-error allows older adults to learn to use smartphone apps by making and correcting mistakes. However, it remains unknown how trial-and-error should be designed to empower older adults to use smartphone apps and how well it would work for older adults. Informed by the guidelines derived from prior work, we designed and implemented ExplorAR, an AR-based trial-and-error system that offers real-time and situated visual guidance in the augmented space around the smartphone to empower older adults to explore and correct mistakes independently. We conducted a user study with 18 older adults to compare ExplorAR with traditional video tutorials and a simplified version of ExplorAR. Results show that the AR-supported trial-and-error method enhanced older adults' learning experience by fostering deeper cognitive engagement and improving confidence in exploring unknown operations.

Problem

• Assisting older adults in learning smartphone apps through AR
• Designing effective trial-and-error methods for elderly users
• Comparing AR guidance with traditional video tutorials

Innovation

• AR-powered trial-and-error learning system
• Real-time situated visual guidance
• Augmented space interactive support