General-purpose LLMs as Models of Human Driver Behavior: The Case of Simplified Merging

2026-03-11

AI Summary

Existing driving behavior models struggle to balance interpretability and flexibility, limiting their generalizability in virtual safety evaluation for autonomous vehicles. This study presents the first systematic validation of off-the-shelf, general-purpose large language models (LLMs)—specifically OpenAI o3 and Google Gemini 2.5 Pro—as plug-and-play human driving behavior models, without requiring fine-tuning. The models are integrated into a one-dimensional merging closed-loop simulation and evaluated through prompt engineering, behavioral comparison, and ablation experiments. Results demonstrate that both LLMs can reproduce key human-like characteristics, such as intermittent control and tactical reliance on spatial cues. However, they exhibit significant and inconsistent deficiencies in dynamic speed response and safety performance, revealing model-specific inductive biases inherent in prompt engineering approaches.

Abstract

Human behavior models are essential as behavior references and for simulating human agents in virtual safety assessment of automated vehicles (AVs), yet current models face a trade-off between interpretability and flexibility. General-purpose large language models (LLMs) offer a promising alternative: a single model potentially deployable without parameter fitting across diverse scenarios. However, what LLMs can and cannot capture about human driving behavior remains poorly understood. We address this gap by embedding two general-purpose LLMs (OpenAI o3 and Google Gemini 2.5 Pro) as standalone, closed-loop driver agents in a simplified one-dimensional merging scenario and comparing their behavior against human data using quantitative and qualitative analyses. Both models reproduce human-like intermittent operational control and tactical dependencies on spatial cues. However, neither consistently captures the human response to dynamic velocity cues, and safety performance diverges sharply between models. A systematic prompt ablation study reveals that prompt components act as model-specific inductive biases that do not transfer across LLMs. These findings suggest that general-purpose LLMs could potentially serve as standalone, ready-to-use human behavior models in AV evaluation pipelines, but future research is needed to better understand their failure modes and ensure their validity as models of human driving behavior.

Problem

• human driver behavior
• large language models
• automated vehicles
• behavior modeling
• virtual safety assessment

Innovation

• large language models
• human driver behavior modeling
• autonomous vehicle simulation
• prompt engineering
• closed-loop driving agents