Communication Strategy on Macro-and-Micro Traffic State in Cooperative Deep Reinforcement Learning for Regional Traffic Signal Control

2025-02-18

AI Summary

To address insufficient agent collaboration in regional traffic signal control (RTSC) using multi-agent deep reinforcement learning (MADRL), this paper proposes a macro-micro dual-granularity communication mechanism. The method introduces a graph attention–driven dual-granularity communication module (GA2-Naive/GA2-Aug), which jointly models macro-level intersection states and micro-level lane states for the first time, explicitly encoding lane-changing behaviors to enable cross-intersection state correlation. It formulates a Markovian evolution equation based on store-and-forward queue dynamics and integrates graph attention networks (GAT) with the RegionLight/Regional-DRL frameworks. Extensive experiments on both real-world and synthetic road networks demonstrate that the proposed approach reduces average vehicle delay by 12.7%–19.3%, confirming its robustness and generalizability in large-scale traffic networks.

Technology Category

• Multiagent Systems: Agent Communication
• Planning, Routing, and Scheduling: Planning with Language Models
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Adaptive Traffic Signal Control (ATSC) has become a popular research topic in intelligent transportation systems. Regional Traffic Signal Control (RTSC) using the Multi-agent Deep Reinforcement Learning (MADRL) technique has become a promising approach for ATSC due to its ability to achieve the optimum trade-off between scalability and optimality. Most existing RTSC approaches partition a traffic network into several disjoint regions, followed by applying centralized reinforcement learning techniques to each region. However, the pursuit of cooperation among RTSC agents still remains an open issue and no communication strategy for RTSC agents has been investigated. In this paper, we propose communication strategies to capture the correlation of micro-traffic states among lanes and the correlation of macro-traffic states among intersections. We first justify the evolution equation of the RTSC process is Markovian via a system of store-and-forward queues. Next, based on the evolution equation, we propose two GAT-Aggregated (GA2) communication modules--GA2-Naive and GA2-Aug to extract both intra-region and inter-region correlations between macro and micro traffic states. While GA2-Naive only considers the movements at each intersection, GA2-Aug also considers the lane-changing behavior of vehicles. Two proposed communication modules are then aggregated into two existing novel RTSC frameworks--RegionLight and Regional-DRL. Experimental results demonstrate that both GA2-Naive and GA2-Aug effectively improve the performance of existing RTSC frameworks under both real and synthetic scenarios. Hyperparameter testing also reveals the robustness and potential of our communication modules in large-scale traffic networks.

Problem

• Enhancing cooperation in Multi-agent Deep Reinforcement Learning
• Developing communication strategies for traffic signal control
• Improving performance of regional traffic signal frameworks

Innovation

• GAT-Aggregated communication modules
• capture macro-micro traffic correlations
• enhance Regional Traffic Signal Control