Mapping fNIRS Signals to Agent Performance: Toward Reinforcement Learning from Neural Feedback

2025-11-16

AI Summary

This study addresses the challenge of constructing passive brain–computer interfaces (BCIs) for reinforcement learning (RL) using implicit neural signals captured via functional near-infrared spectroscopy (fNIRS). We propose a novel fNIRS-based continuous performance regression framework that maps prefrontal hemodynamic responses to the degree of agent behavioral deviation from an optimal policy. The method is validated on robotic manipulation and gaming tasks. Our key contributions include: (1) the first publicly released multi-domain, cross-subject fNIRS dataset; (2) a model achieving average F1-scores of 67% (binary) and 46% (multi-class) classification accuracy—improving to 84% and 87%, respectively, after fine-tuning with minimal subject-specific data; and (3) a regression model significantly outperforming baselines, demonstrating that implicit fNIRS signals can support cross-subject generalizable neurofeedback for RL.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Humans and AI: Brain-Sensing and Analysis
• Intelligent Robots: Behavior Learning & Control

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Reinforcement Learning from Human Feedback (RLHF) is a methodology that aligns agent behavior with human preferences by integrating human feedback into the agent's training process. We introduce a possible framework that employs passive Brain-Computer Interfaces (BCI) to guide agent training from implicit neural signals. We present and release a novel dataset of functional near-infrared spectroscopy (fNIRS) recordings collected from 25 human participants across three domains: a Pick-and-Place Robot, Lunar Lander, and Flappy Bird. We train classifiers to predict levels of agent performance (optimal, sub-optimal, or worst-case) from windows of preprocessed fNIRS feature vectors, achieving an average F1 score of 67% for binary classification and 46% for multi-class models averaged across conditions and domains. We also train regressors to predict the degree of deviation between an agent's chosen action and a set of near-optimal policies, providing a continuous measure of performance. We evaluate cross-subject generalization and demonstrate that fine-tuning pre-trained models with a small sample of subject-specific data increases average F1 scores by 17% and 41% for binary and multi-class models, respectively. Our work demonstrates that mapping implicit fNIRS signals to agent performance is feasible and can be improved, laying the foundation for future brain-driven RLHF systems.

Problem

• Mapping fNIRS neural signals to agent performance metrics
• Training classifiers to predict agent performance from brain data
• Developing brain-driven reinforcement learning from neural feedback

Innovation

• Using fNIRS signals to guide agent training
• Training classifiers to predict agent performance levels
• Fine-tuning models with subject-specific data improves accuracy