When Context Dominates: Multimodal Signatures of Takeover Readiness Under Varying Hazard and Cognitive Load Conditions

2026-07-12

AI Summary

This study addresses the unclear joint influence of hazardous scenarios and cognitive load on driver takeover readiness in semi-autonomous driving. Through a driving simulator experiment, it integrates multimodal data—including vehicle dynamics, subjective workload ratings, autonomic nervous system measures (electrodermal activity and heart rate variability), and functional near-infrared spectroscopy—to systematically examine how these factors shape takeover behavior. Results indicate that hazard type is the dominant factor: unexpected pedestrian hazards elicit longer and more variable takeover responses, whereas static collision threats prompt faster, more stable reactions. Although secondary tasks exert limited effects on vehicle control, they significantly modulate physiological and neural markers. The work proposes a new paradigm for assessing takeover readiness by jointly considering external situational demands and internal cognitive states.

Abstract

Semi-automated driving systems promise to reduce crashes by assisting with perception and control, yet they simultaneously introduce additional human factors challenges by requiring drivers to monitor automation and rapidly resume control when failures occur. Prolonged passive monitoring can degrade vigilance, delay reactions, and increase takeover risk, but the extent to which distraction, hazard context, and drivers' underlying cognitive and physiological states jointly shape takeover performance remains insufficiently understood. This study investigates these interacting factors using a controlled, within-subjects driving simulator experiment that crosses two hazard types (dynamic pedestrian and static crash events) with three levels of secondary task engagement (no task, conversation, and working memory load). Driver responses were assessed using a multimodal sensing framework that integrates vehicle-dynamics measures, subjective workload ratings, autonomic physiology (electrodermal activity and heart rate variability), and prefrontal cortical activation measured with functional near-infrared spectroscopy. Results show that hazard context is the primary determinant of takeover behavior, with pedestrian events producing longer and more variable maneuvers and crash events yielding faster and more stable responses. Secondary tasks exerted smaller effects on objective vehicle control, while internal-state measures showed more variable task-related patterns. These findings highlight the importance of jointly considering environmental context and human state when evaluating takeover readiness and designing driver monitoring systems. This study lays the groundwork for adaptive, context-aware strategies that support safer human-automation collaboration in semi-automated vehicles.

Problem

• takeover readiness
• hazard context
• cognitive load
• driver state
• semi-automated driving

Innovation

• multimodal sensing
• takeover readiness
• hazard context
• cognitive load
• functional near-infrared spectroscopy