Towards Interactive and Learnable Cooperative Driving Automation: A Large Language Model-Driven Decision-Making Framework

2024-09-19

IEEE Transactions on Vehicular Technology

AI Summary

Current collaborative driving systems suffer from scene-specific adaptation, static assignment of Cooperative Driving Automation (CDA) levels, and insufficient real-time interaction and continual learning capabilities—hindering robust, multi-level cooperative decision-making across diverse scenarios. To address these limitations, this paper proposes CoDrivingLLM: a novel framework featuring a coupled reasoning architecture that integrates centralized negotiation with distributed decision-making. Leveraging large language models (LLMs) and semantic-driven modeling, it enables SAE J3216-compliant four-level CDA inference. Additionally, a Retrieval-Augmented Generation (RAG)-enhanced memory module supports online retrieval and continual evolution of driving experience. Experimental results demonstrate significant improvements in safety, traffic efficiency, and environmental adaptability across multiple scenarios in ablation studies, thereby overcoming key bottlenecks of existing approaches in interactivity, generalizability, and evolutionary capability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Connected Autonomous Vehicles (CAVs) are being tested globally, but their performance in complex scenarios remains suboptimal. While cooperative driving improves CAV performance by leveraging vehicle collaboration, its lack of interaction and continuous learning limits current applications to single scenarios and specific Cooperative Driving Automation (CDA). To address these issues, this paper proposes CoDrivingLLM, an interactive and learnable LLM-driven cooperative driving framework for all-scenario and all-CDA applications. First, an environment module updates vehicle positions based on semantic decisions, mitigating errors from LLM-controlled positioning. Second, leveraging the four CDA levels defined in SAE J3216, a centralized-distributed coupled architecture reasoning module is developed to ensure safe and efficient cooperation through centralized negotiation and distributed decision. Finally, by introducing a memory module that employs Retrieval Augmented Generation (RAG), CAVs are endowed with the ability to learn from their past experiences to avoid repeating mistakes. Through ablation studies and comparisons with other cooperative driving methods, the results demonstrate that the proposed CoDrivingLLM significantly enhances safety, efficiency, and adaptability across various scenarios.

Problem

• Enhancing CAV safety and efficiency in complex scenarios
• Overcoming lack of interaction in cooperative driving automation
• Enabling continuous learning for all-scenario CDA applications

Innovation

• Environment module updates vehicle positions semantically
• Chain-of-Thought reasoning for multi-step decision-making
• Memory module enables experience-based learning capability