Through the Expert's Eyes: Exploring Asynchronous Expert Perspectives and Gaze Visualizations in XR

2025-08-31

AI Summary

This study addresses spatial representation distortion and weakened social connectivity arising from the absence of on-site guidance in intergenerational transmission of complex practical skills. We propose an asynchronous instructional framework integrating extended reality (XR), eye-tracking, and artificial intelligence. The framework innovatively fuses first-person perspective (FPP), third-person perspective (TPP), and expert gaze trajectory visualization to jointly enhance spatial cognitive accuracy and embodied presence. Experimental results show that FPP significantly outperforms other conditions in procedural accuracy, learning efficiency, and subjective preference; gaze visualization effectively compensates for intent ambiguity inherent in asynchronous interaction. Based on empirical findings, we derive XR-based multi-perspective coordination design principles and implementation guidelines tailored to practical knowledge transfer. This work provides a reusable technical pathway and theoretical foundation for remote skill transmission.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Computer Vision: Motion & Tracking

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Transferring knowledge across generations is fundamental to human civilization, yet the challenge of passing on complex practical skills persists. Methods without a physically present instructor, such as videos, often fail to explain complex manual tasks, where spatial and social factors are critical. Technologies such as eXtended Reality and Artificial Intelligence hold the potential to retain expert knowledge and facilitate the creation of tailored, contextualized, and asynchronous explanations regardless of time and place. In contrast to videos, the learner's perspective can be different from the recorded perspective in XR. This paper investigates the impact of asynchronous first- and third-person perspectives and gaze visualizations on efficiency, feeling of embodiment, and connectedness during manual tasks. The empirical results of our study (N=36) show that the first-person perspective is better in quantitative measures and preferred by users. We identify best practices for presenting preserved knowledge and provide guidelines for designing future systems.

Problem

• Transferring complex practical skills asynchronously
• Evaluating first- vs third-person perspectives in XR
• Assessing impact of gaze visualizations on learning

Innovation

• First-person perspective in XR training
• Gaze visualization for skill transfer
• Asynchronous expert knowledge preservation