Steering through Time: Blending Longitudinal Data with Simulation to Rethink Human-Autonomous Vehicle Interaction

2026-04-01

AI Summary

This study addresses the lack of temporal context regarding drivers’ cognitive and physiological states during human–automation handover in semi-autonomous driving. To this end, it proposes a multimodal hybrid framework that integrates longitudinal observational data with real-time driving simulation. Over a seven-day period, the study collected wearable-derived physiological measures (including HRV and RMSSD), daily psychological surveys, eye-tracking metrics, functional near-infrared spectroscopy (fNIRS), and high-fidelity driving simulator data to establish, for the first time, a temporally deep, personalized paradigm for monitoring driver state. Results demonstrate that takeover performance—such as gaze duration and takeover time—is significantly influenced by secondary task type, while RMSSD exhibits high inter-individual stability. These findings validate the feasibility of the proposed framework and offer a novel pathway toward personalized, context-aware assessment of takeover readiness.

Abstract

As semi-automated vehicles (SAVs) become more common, ensuring effective human-vehicle interaction during control handovers remains a critical safety challenge. Existing studies often rely on single-session simulator experiments or naturalistic driving datasets, which often lack temporal context on drivers' cognitive and physiological states before takeover events. This study introduces a hybrid framework combining longitudinal mobile sensing with high-fidelity driving simulation to examine driver readiness in semi-automated contexts. In a pilot study with 38 participants, we collected 7 days of wearable physiological data and daily surveys on stress, arousal, valence, and sleep quality, followed by an in-lab simulation with scripted takeover events under varying secondary task conditions. Multimodal sensing, including eye tracking, fNIRS, and physiological measures, captured real-time responses. Preliminary analysis shows the framework's feasibility and individual variability in baseline and in-task measures; for example, fixation duration and takeover control time differed by task type, and RMSSD showed high inter-individual stability. This proof-of-concept supports the development of personalized, context-aware driver monitoring by linking temporally layered data with real-time performance.

Problem

• human-autonomous vehicle interaction
• takeover readiness
• longitudinal data
• driver monitoring
• semi-automated vehicles

Innovation

• longitudinal sensing
• human-autonomous vehicle interaction
• takeover readiness
• multimodal physiological monitoring
• driving simulation