Multi-agent DRL-based Lane Change Decision Model for Cooperative Planning in Mixed Traffic

2026-01-16

AI Summary

This study addresses the challenge of inefficient cooperative platooning in early-stage mixed traffic, where connected and automated vehicles (CAVs) suffer from low market penetration. To overcome this limitation, the authors propose a lane-changing decision-making model based on multi-agent deep reinforcement learning, featuring an innovative CNN-QMIX architecture. This framework integrates convolutional neural networks to process dynamic traffic states and couples with a trajectory planner and a model predictive controller to ensure safe and smooth lane changes. The approach enables scalable coordination among a variable number of CAVs and demonstrates significant performance gains over rule-based baselines across diverse penetration rates, achieving a maximum improvement of 26.2% in cooperative platooning efficiency.

Technology Category

• Planning, Routing, and Scheduling: Mixed Discrete/Continuous Planning
• Multiagent Systems: Multiagent Planning
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Connected automated vehicles (CAVs) possess the ability to communicate and coordinate with one another, enabling cooperative platooning that enhances both energy efficiency and traffic flow. However, during the initial stage of CAV deployment, the sparse distribution of CAVs among human-driven vehicles reduces the likelihood of forming effective cooperative platoons. To address this challenge, this study proposes a hybrid multi-agent lane change decision model aimed at increasing CAV participation in cooperative platooning and maximizing its associated benefits. The proposed model employs the QMIX framework, integrating traffic data processed through a convolutional neural network (CNN-QMIX). This architecture addresses a critical issue in dynamic traffic scenarios by enabling CAVs to make optimal decisions irrespective of the varying number of CAVs present in mixed traffic. Additionally, a trajectory planner and a model predictive controller are designed to ensure smooth and safe lane-change execution. The proposed model is trained and evaluated within a microsimulation environment under varying CAV market penetration rates. The results demonstrate that the proposed model efficiently manages fluctuating traffic agent numbers, significantly outperforming the baseline rule-based models. Notably, it enhances cooperative platooning rates up to 26.2\%, showcasing its potential to optimize CAV cooperation and traffic dynamics during the early stage of deployment.

Problem

• multi-agent
• lane change
• cooperative platooning
• mixed traffic
• CAVs

Innovation

• multi-agent reinforcement learning
• cooperative platooning
• QMIX
• mixed traffic
• lane change decision