Neuro-Cognitive Reward Modeling for Human-Centered Autonomous Vehicle Control

2026-03-26

AI Summary

This work proposes a novel EEG-guided reinforcement learning framework to enhance the alignment of autonomous driving systems with human driving intent, addressing the inefficiency and indirectness of conventional approaches that rely on manually annotated preference data. By directly incorporating event-related potentials (ERPs)—neural responses elicited during unexpected driving scenarios in simulation—into the reinforcement learning reward function, the method operates without behavioral interruption or human annotation. The framework establishes an end-to-end neurocognitive feedback loop by simultaneously acquiring EEG signals, extracting visual scene features, and using neural networks to predict ERP amplitudes. Experimental results demonstrate that this approach significantly improves the agent’s ability to avoid collisions in sudden, high-risk situations, thereby validating the efficacy of leveraging neurocognitive signals for efficient and natural human–agent alignment.

Abstract

Recent advancements in computer vision have accelerated the development of autonomous driving. Despite these advancements, training machines to drive in a way that aligns with human expectations remains a significant challenge. Human factors are still essential, as humans possess a sophisticated cognitive system capable of rapidly interpreting scene information and making accurate decisions. Aligning machine with human intent has been explored with Reinforcement Learning with Human Feedback (RLHF). Conventional RLHF methods rely on collecting human preference data by manually ranking generated outputs, which is time-consuming and indirect. In this work, we propose an electroencephalography (EEG)-guided decision-making framework to incorporate human cognitive insights without behaviour response interruption into reinforcement learning (RL) for autonomous driving. We collected EEG signals from 20 participants in a realistic driving simulator and analyzed event-related potentials (ERP) in response to sudden environmental changes. Our proposed framework employs a neural network to predict the strength of ERP based on the cognitive information from visual scene information. Moreover, we explore the integration of such cognitive information into the reward signal of the RL algorithm. Experimental results show that our framework can improve the collision avoidance ability of the RL algorithm, highlighting the potential of neuro-cognitive feedback in enhancing autonomous driving systems. Our project page is: https://alex95gogo.github.io/Cognitive-Reward/.

Problem

• human-centered autonomous driving
• cognitive alignment
• neuro-cognitive feedback
• reinforcement learning
• human intent

Innovation

• EEG-guided reinforcement learning
• neuro-cognitive reward
• event-related potentials (ERP)
• human-centered autonomous driving
• cognitive feedback