Will You Be Aware? Eye Tracking-Based Modeling of Situational Awareness in Augmented Reality

2025-08-07

AI Summary

This study addresses the degradation of situational awareness (SA) in augmented reality (AR)-assisted cardiopulmonary resuscitation (CPR) caused by cognitive tunneling. To this end, we propose a dynamic attention modeling method based on eye-tracking data. We introduce FixGraphPool, a novel graph neural network that models gaze events as spatiotemporal graphs and incorporates domain-knowledge-guided graph pooling to jointly encode temporal and spatial features of eye movements for real-time SA prediction. Experimental results demonstrate that FixGraphPool achieves 83.0% SA prediction accuracy (F1-score = 81.0%), significantly outperforming conventional machine learning and sequential models. The work validates the efficacy of eye-tracking–driven attention modeling in enhancing AR system safety and establishes an interpretable, deployable paradigm for human factors design in high-risk AR applications.

Technology Category

• Cognitive Modeling & Cognitive Systems: Affective Computing
• Computer Vision: Motion & Tracking
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Augmented Reality (AR) systems, while enhancing task performance through real-time guidance, pose risks of inducing cognitive tunneling-a hyperfocus on virtual content that compromises situational awareness (SA) in safety-critical scenarios. This paper investigates SA in AR-guided cardiopulmonary resuscitation (CPR), where responders must balance effective compressions with vigilance to unpredictable hazards (e.g., patient vomiting). We developed an AR app on a Magic Leap 2 that overlays real-time CPR feedback (compression depth and rate) and conducted a user study with simulated unexpected incidents (e.g., bleeding) to evaluate SA, in which SA metrics were collected via observation and questionnaires administered during freeze-probe events. Eye tracking analysis revealed that higher SA levels were associated with greater saccadic amplitude and velocity, and with reduced proportion and frequency of fixations on virtual content. To predict SA, we propose FixGraphPool, a graph neural network that structures gaze events (fixations, saccades) into spatiotemporal graphs, effectively capturing dynamic attentional patterns. Our model achieved 83.0% accuracy (F1=81.0%), outperforming feature-based machine learning and state-of-the-art time-series models by leveraging domain knowledge and spatial-temporal information encoded in ET data. These findings demonstrate the potential of eye tracking for SA modeling in AR and highlight its utility in designing AR systems that ensure user safety and situational awareness.

Problem

• Investigates situational awareness risks in AR-guided CPR tasks
• Develops eye-tracking model to predict user awareness in AR
• Proposes FixGraphPool to improve safety in AR systems

Innovation

• AR app with real-time CPR feedback
• Eye tracking for situational awareness metrics
• FixGraphPool model predicts awareness accurately