Gaze-Hand Steering for Travel and Multitasking in Virtual Environments

2025-04-02

AI Summary

To address gaze-based navigation’s susceptibility to false triggers and hand-controller interaction’s field-of-view occlusion in VR, this paper proposes a gaze-hand steering technique: navigation is triggered only when both the user’s gaze direction and the 6-DoF orientation of the handheld controller jointly align with the target—enabling seamless visual exploration and precise directional control. This work introduces the first gaze-hand co-triggering mechanism, integrating HMD-based eye tracking with dual-mode motion input—either a waist-mounted velocity ring or a standard VR controller—and evaluates it via a 20-participant user study. Results demonstrate maintained task accuracy across multi-task scenarios, alongside statistically significant improvements in subjective comfort and spatial awareness (p < 0.01). The approach effectively balances navigation performance with multi-task compatibility, establishing a novel paradigm for highly immersive, high-precision, and multi-task-capable VR interaction.

Technology Category

• Computer Vision: Multi-modal Vision
• Humans and AI: Interaction Techniques and Devices
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

As head-mounted displays (HMDs) with eye-tracking become increasingly accessible, the need for effective gaze-based interfaces in virtual reality (VR) grows. Traditional gaze- or hand-based navigation often limits user precision or impairs free viewing, making multitasking difficult. We present a gaze-hand steering technique that combines eye-tracking with hand-pointing: users steer only when gaze aligns with a hand-defined target, reducing unintended actions and enabling free look. Speed is controlled via either a joystick or a waist-level speed circle. We evaluated our method in a user study (N=20) across multitasking and single-task scenarios, comparing it to a similar technique. Results show that gaze-hand steering maintains performance and enhances user comfort and spatial awareness during multitasking. Our findings support the use of gaze-hand steering in gaze-dominant VR applications requiring precision and simultaneous interaction. Our method significantly improves VR navigation in gaze-dominant, multitasking-intensive applications, supporting immersion and efficient control.

Problem

• Combines eye-tracking with hand-pointing for precise VR navigation
• Reduces unintended actions and enables free viewing in VR
• Enhances user comfort and spatial awareness during multitasking

Innovation

• Combines eye-tracking with hand-pointing for steering
• Controls speed via joystick or waist-level circle
• Enhances precision and multitasking in VR