Passenger hazard perception based on EEG signals for highly automated driving vehicles

2024-08-29

arXiv.org

AI Summary

Enhancing safety for high-level autonomous vehicles necessitates proactive hazard detection beyond conventional sensor-based approaches. Method: This study proposes a novel pre-emptive hazard perception paradigm leveraging passive, task-free passenger electroencephalography (EEG). We develop a Passenger Cognitive Model (PCM) and an EEG Decoding Strategy (PEDS), featuring a custom spatiotemporal convolutional recurrent neural network (CRNN) integrated with stacked ensemble learning to decode predictive neural signatures from EEG signals. Critically, we identify statistically significant pre-event EEG biomarkers occurring several seconds prior to hazardous events. Contribution/Results: Our framework achieves 85.0% ± 3.18% classification accuracy in hazard-scene identification—demonstrating, for the first time, the feasibility and efficacy of using unobtrusive, task-free passenger EEG for prospective neuro-alerting. This work establishes a theoretically grounded and practically implementable pathway toward human–vehicle neural interaction–enhanced safety architectures in autonomous driving.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Hardware-aware ML

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Large-scale security measurements
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Enhancing the safety of autonomous vehicles is crucial, especially given recent accidents involving automated systems. As passengers in these vehicles, humans' sensory perception and decision-making can be integrated with autonomous systems to improve safety. This study explores neural mechanisms in passenger-vehicle interactions, leading to the development of a Passenger Cognitive Model (PCM) and the Passenger EEG Decoding Strategy (PEDS). Central to PEDS is a novel Convolutional Recurrent Neural Network (CRNN) that captures spatial and temporal EEG data patterns. The CRNN, combined with stacking algorithms, achieves an accuracy of $85.0% pm 3.18%$. Our findings highlight the predictive power of pre-event EEG data, enhancing the detection of hazardous scenarios and offering a network-driven framework for safer autonomous vehicles.

Problem

• Enhancing autonomous vehicle safety using passenger EEG signals
• Developing neural network models for hazard perception prediction
• Improving hazard detection accuracy with brain activity analysis

Innovation

• CRNN captures spatial-temporal EEG patterns
• Stacking algorithms boost decoding accuracy
• Pre-event EEG predicts hazardous scenarios