NeuGaze: Reshaping the future BCI

2025-04-21

AI Summary

Traditional brain–computer interfaces (BCIs) rely on expensive EEG hardware or invasive implants, suffering from complex deployment, low spatial resolution, and time-consuming user calibration. This paper introduces NeuGaze—the first end-to-end, non-invasive visual BCI system that operates solely on a standard 30-Hz webcam. Leveraging a lightweight multi-task spatiotemporal convolutional network, NeuGaze jointly models gaze direction, head pose, and facial action units (AUs) to enable calibration-free, sub-second, pixel-accurate cursor control. Its key contributions are: (1) the first webcam-based unified modeling of gaze, head, and face dynamics for BCI; (2) support for skill-hotkey activation and real-time interaction in FPS games; and (3) demonstration of >89% win rate against human players on commodity laptops, with a 99% reduction in deployment cost versus EEG-based BCIs and calibration completed in under 30 seconds.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Humans and AI: Brain-Sensing and Analysis

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Browser and end-point security
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Traditional brain-computer interfaces (BCIs), reliant on costly electroencephalography or invasive implants, struggle with complex human-computer interactions due to setup complexity and limited precision. We present NeuGaze, a novel webcam-based system that leverages eye gaze, head movements, and facial expressions to enable intuitive, real-time control using only a standard 30 Hz webcam, often pre-installed in laptops. Requiring minimal calibration, NeuGaze achieves performance comparable to conventional inputs, supporting precise cursor navigation, key triggering via an efficient skill wheel, and dynamic gaming interactions, such as defeating formidable opponents in first-person games. By harnessing preserved neck-up functionalities in motor-impaired individuals, NeuGaze eliminates the need for specialized hardware, offering a low-cost, accessible alternative to BCIs. This paradigm empowers diverse applications, from assistive technology to entertainment, redefining human-computer interaction for motor-impaired users. Project is at href{https://github.com/NeuSpeech/NeuGaze}{github.com/NeuSpeech/NeuGaze}.

Problem

• Replacing costly BCIs with affordable webcam-based system
• Enabling intuitive control via eye gaze and facial expressions
• Providing accessible interaction for motor-impaired individuals

Innovation

• Webcam-based system using eye gaze and head movements
• Minimal calibration with real-time, intuitive control
• Low-cost alternative to traditional brain-computer interfaces