Large-Scale Mixed-Traffic and Intersection Control using Multi-agent Reinforcement Learning

2025-04-07

AI Summary

Coordinating mixed urban traffic—comprising traffic signals and autonomous robotic vehicles—at scale remains challenging due to heterogeneity, spatial coupling, and scalability limitations. Method: This paper introduces the first decentralized multi-agent reinforcement learning (MARL) control framework designed for real-world road networks. It models heterogeneous traffic entities and leverages digital twin simulation to enable joint, distributed optimization of traffic signals and robotic vehicles across a 14-intersection real-world network. Contribution/Results: Unlike prior isolated-intersection approaches, our framework achieves scalable, robust control under high vehicle automation penetration. At 80% robotic vehicle penetration, it reduces average vehicle waiting time by 17.5% (from 6.17 s to 5.09 s) and increases throughput by 8.6% (from 454 to 493 vehicles within 500 seconds). This work establishes a new paradigm for adaptive, large-scale traffic management in highly connected and automated environments.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Intelligent Robots: Multi-Robot Systems
• Multiagent Systems: Multiagent Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Traffic congestion remains a significant challenge in modern urban networks. Autonomous driving technologies have emerged as a potential solution. Among traffic control methods, reinforcement learning has shown superior performance over traffic signals in various scenarios. However, prior research has largely focused on small-scale networks or isolated intersections, leaving large-scale mixed traffic control largely unexplored. This study presents the first attempt to use decentralized multi-agent reinforcement learning for large-scale mixed traffic control in which some intersections are managed by traffic signals and others by robot vehicles. Evaluating a real-world network in Colorado Springs, CO, USA with 14 intersections, we measure traffic efficiency via average waiting time of vehicles at intersections and the number of vehicles reaching their destinations within a time window (i.e., throughput). At 80% RV penetration rate, our method reduces waiting time from 6.17 s to 5.09 s and increases throughput from 454 vehicles per 500 seconds to 493 vehicles per 500 seconds, outperforming the baseline of fully signalized intersections. These findings suggest that integrating reinforcement learning-based control large-scale traffic can improve overall efficiency and may inform future urban planning strategies.

Problem

• Large-scale mixed traffic control with multi-agent reinforcement learning
• Integrating autonomous vehicles and traffic signals for urban efficiency
• Reducing vehicle waiting time and increasing throughput in intersections

Innovation

• Decentralized multi-agent reinforcement learning for traffic
• Mixed control with signals and robot vehicles
• Improved efficiency in large-scale urban networks