🤖 AI Summary
This study presents the first systematic evaluation of the visual and cognitive workload induced by a large language model (LLM)-driven in-vehicle voice agent (Gemini Live) during real-world driving. Through an on-road multitasking experiment, the authors employed the Detection Response Task (DRT), eye-tracking metrics, and subjective workload ratings to compare the distraction levels of Gemini Live against baseline conditions including hands-free phone calls, navigation tasks, and a high-demand memory task. Results indicate that Gemini Live imposes a cognitive load comparable to hands-free calling and intermediate between low-demand navigation and high-demand memory tasks. Its visual demand remains significantly below the 2-second safety threshold, and users report low subjective interference, collectively demonstrating a safe operational boundary for LLM-based voice agents in automotive contexts and providing empirical support for their secure deployment.
📝 Abstract
Driver distraction remains a leading contributor to motor vehicle crashes, necessitating rigorous evaluation of new in-vehicle technologies. This study assessed the visual and cognitive demands associated with an advanced Large Language Model (LLM) conversational agent (Gemini Live) during on-road driving, comparing it against handsfree phone calls, visual turn-by-turn guidance (low load baseline), and the Operation Span (OSPAN) task (high load anchor). Thirty-two licensed drivers completed five secondary tasks while visual and cognitive demands were measured using the Detection Response Task (DRT) for cognitive load, eye-tracking for visual attention, and subjective workload ratings. Results indicated that Gemini Live interactions (both single-turn and multi-turn) and hands-free phone calls shared similar levels of cognitive load, between that of visual turn-by-turn guidance and OSPAN. Exploratory analysis showed that cognitive load remained stable across extended multi-turn conversations. All tasks maintained mean glance durations well below the well-established 2-second safety threshold, confirming low visual demand. Furthermore, drivers consistently dedicated longer glances to the roadway between brief off-road glances toward the device during task completion, particularly during voice-based interactions, rendering longer total-eyes-off-road time findings less consequential. Subjective ratings mirrored objective data, with participants reporting low effort, demands, and perceived distraction for Gemini Live. These findings demonstrate that advanced LLM conversational agents, when implemented via voice interfaces, impose cognitive and visual demands comparable to established, low-risk hands-free benchmarks, supporting their safe deployment in the driving environment.