A Real-Time Error Prevention System for Gaze-Based Interaction in Virtual Reality Based on Anomaly Detection

2026-01-21

AI Summary

This work addresses the susceptibility of eye-gaze interaction in virtual reality to unintended inputs, which frequently leads to erroneous selections. To mitigate this issue, the authors propose a real-time Error Prevention System (EPS) that introduces an anomaly-detection mechanism into gaze-based interaction for the first time. Leveraging a Temporal Convolutional Autoencoder (TCNAE), EPS identifies anomalous dynamics during eye-gaze selection tasks and intercepts potential errors without disrupting the user’s natural interaction flow. The system integrates three interaction paradigms—gaze dwell, eye-head alignment, and head nodding—and achieves up to a 95% reduction in erroneous selections within a VR environment. User evaluations yielded positive subjective feedback, demonstrating the system’s strong potential for applications in VR/AR and assistive technologies.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Humans and AI: Interaction Techniques and Devices
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

Gaze-based interaction enables intuitive, hands-free control in immersive environments, but remains susceptible to unintended inputs. We present a real-time error prevention system (EPS) that uses a temporal convolutional network autoencoder (TCNAE) to detect anomalies in gaze dynamics during selection tasks. In a visual search task in VR, 41 participants used three gaze-based methods - dwell time, gaze and head direction alignment, and nod - with and without EPS. The system reduced erroneous selections by up to 95% for dwell time and gaze and head, and was positively received by most users. Performance varied for nodding and between individuals, suggesting the need for adaptive systems. Objective metrics and subjective evaluations show that anomaly-based error prevention can improve gaze interfaces without disrupting interaction. These findings demonstrate the potential of anomaly-based error prevention for gaze interfaces and suggest applications in VR, AR, and assistive technologies.

Problem

• gaze-based interaction
• error prevention
• unintended inputs
• virtual reality
• anomaly detection

Innovation

• anomaly detection
• temporal convolutional network autoencoder
• gaze-based interaction
• error prevention
• virtual reality