An offline approach to fNIRS-guided reinforcement learning for robot behavior

2026-07-15

AI Summary

This work proposes a novel framework for offline functional near-infrared spectroscopy (fNIRS)-guided robot reinforcement learning, designed for scenarios where real-time brain–computer interfaces are impractical. The approach collects human neural signals through passive observation and active demonstration tasks, then integrates these fNIRS-derived features into offline reinforcement learning via parameter augmentation. Specifically, the neural signals modulate trajectory prioritization in experience replay and refine state–action Q-value estimation. Experimental results demonstrate that incorporating fNIRS signals substantially enhances both learning efficiency and policy performance of the agent, thereby validating the feasibility and effectiveness of offline neuro-guidance in human–robot interaction.

Abstract

Human-in-the-loop Reinforcement Learning has become a popular approach to training, finetuning, and aligning robot behavior with user preferences. Our paper explores the feasibility of using brain signals via functional near-infrared spectroscopy (fNIRS) to modulate robot learning in simulation. We compare agents trained on passive (observational) versus active (demonstrative) interaction tasks, and test multiple methods for enhancing the RL algorithm with the neural signal, focusing on parameter augmentation rather than replacement. We further examine how model granularity and noise affect agent learning. Our results show that this framework is effective: the neural signal improves learning when augmenting trajectory priorities and state-action q-values. Additionally, the framework learns successfully from offline data, offering a practical alternative for settings where real-time BCI setups are impractical or only limited data is available.

Problem

• fNIRS
• reinforcement learning
• robot behavior
• offline learning
• human-in-the-loop

Innovation

• fNIRS
• offline reinforcement learning
• human-in-the-loop
• neural signal augmentation
• robot behavior alignment