Driver Assistant: Persuading Drivers to Adjust Secondary Tasks Using Large Language Models

2025-08-07

AI Summary

In Level 3 automated driving, drivers’ engagement in secondary tasks compromises takeover readiness and overburdens them with alarm responses. To address this, we propose a multimodal, human-centered persuasion mechanism powered by large language models (LLMs). The system dynamically perceives driving context and driver state, and proactively delivers anthropomorphic visual–auditory cues prior to critical takeovers to guide attention allocation. Its key innovation lies in the first integration of LLMs into in-vehicle human–machine interaction, enabling context-adaptive, semantically natural, and proactive persuasion. Experimental results demonstrate significant improvements: a 28.6% increase in road gaze rate, a 31.4% reduction in subjective cognitive load (measured via NASA-TLX), and a 1.37-second decrease in mean takeover reaction time. These outcomes substantiate enhanced takeover reliability and more natural, intuitive interaction.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Level 3 automated driving systems allows drivers to engage in secondary tasks while diminishing their perception of risk. In the event of an emergency necessitating driver intervention, the system will alert the driver with a limited window for reaction and imposing a substantial cognitive burden. To address this challenge, this study employs a Large Language Model (LLM) to assist drivers in maintaining an appropriate attention on road conditions through a "humanized" persuasive advice. Our tool leverages the road conditions encountered by Level 3 systems as triggers, proactively steering driver behavior via both visual and auditory routes. Empirical study indicates that our tool is effective in sustaining driver attention with reduced cognitive load and coordinating secondary tasks with takeover behavior. Our work provides insights into the potential of using LLMs to support drivers during multi-task automated driving.

Problem

• Persuading drivers to adjust secondary tasks during automated driving
• Reducing cognitive burden in emergency driver interventions
• Maintaining driver attention with humanized LLM advice

Innovation

• LLM provides humanized persuasive advice
• Visual and auditory cues guide behavior
• Reduces cognitive load during driving