AdaptiveCoPilot: Design and Testing of a NeuroAdaptive LLM Cockpit Guidance System in both Novice and Expert Pilots

2025-01-07

AI Summary

Cognitive overload from multitasking in modern cockpits jeopardizes flight safety. To address this, we propose the first neuroadaptive large language model (LLM)-guided system tailored for aviation: it fuses real-time functional near-infrared spectroscopy (fNIRS) neural signals with behavioral data to construct a dual-driven cognitive load model—simultaneously quantifying working memory and perceptual load. We introduce the first deep integration of fNIRS-based neural feedback with context-aware LLMs, enabling dynamic, load-informed modality switching among visual, auditory, and textual prompts. In a VR-based flight simulation experiment with licensed pilots, the system significantly increased the proportion of time spent within the optimal cognitive load zone (p < 0.01), reduced task completion time, and achieved statistically significant improvements in both working memory and perceptual load regulation (p < 0.01). These results empirically validate the efficacy of our neuro-semantic co-adaptive paradigm for adaptive human–autonomy interaction in safety-critical domains.

Technology Category

• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Natural Language Processing: Safety and Robustness
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Pilots operating modern cockpits often face high cognitive demands due to complex interfaces and multitasking requirements, which can lead to overload and decreased performance. This study introduces AdaptiveCoPilot, a neuroadaptive guidance system that adapts visual, auditory, and textual cues in real time based on the pilot's cognitive workload, measured via functional Near-Infrared Spectroscopy (fNIRS). A formative study with expert pilots (N=3) identified adaptive rules for modality switching and information load adjustments during preflight tasks. These insights informed the design of AdaptiveCoPilot, which integrates cognitive state assessments, behavioral data, and adaptive strategies within a context-aware Large Language Model (LLM). The system was evaluated in a virtual reality (VR) simulated cockpit with licensed pilots (N=8), comparing its performance against baseline and random feedback conditions. The results indicate that the pilots using AdaptiveCoPilot exhibited higher rates of optimal cognitive load states on the facets of working memory and perception, along with reduced task completion times. Based on the formative study, experimental findings, qualitative interviews, we propose a set of strategies for future development of neuroadaptive pilot guidance systems and highlight the potential of neuroadaptive systems to enhance pilot performance and safety in aviation environments.

Problem

• Cockpit Design
• Cognitive Load
• Flight Safety

Innovation

• AdaptiveCoPilot
• CognitiveLoadReduction
• FlightEfficiency