Large-scale Regional Traffic Signal Control Based on Single-Agent Reinforcement Learning

2025-03-12

AI Summary

To address large-scale urban traffic congestion, this paper proposes a single-agent deep reinforcement learning framework—employing PPO or DQN—for coordinated traffic signal control across multiple intersections. Methodologically, we design a dynamic phase-splitting mechanism and custom state/action spaces, implemented within the SUMO microscopic traffic simulator; a dual-objective reward function jointly optimizes queue length and total travel time. Our key contribution is the first adoption of a single-agent architecture for city-wide signal control, eliminating inter-agent communication overhead and mitigating multi-agent convergence challenges, while enabling global optimization beyond local control constraints. Experiments demonstrate that, compared to fixed-time signal plans, our approach reduces average travel time by 23.6% and queue length by 31.4%, significantly improving regional traffic efficiency. These results validate the method’s effectiveness and scalability for large-scale deployment.

Technology Category

• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Planning/Scheduling and Learning
• Multiagent Systems: Multiagent Learning

Application Category

• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In the context of global urbanization and motorization, traffic congestion has become a significant issue, severely affecting the quality of life, environment, and economy. This paper puts forward a single-agent reinforcement learning (RL)-based regional traffic signal control (TSC) model. Different from multi - agent systems, this model can coordinate traffic signals across a large area, with the goals of alleviating regional traffic congestion and minimizing the total travel time. The TSC environment is precisely defined through specific state space, action space, and reward functions. The state space consists of the current congestion state, which is represented by the queue lengths of each link, and the current signal phase scheme of intersections. The action space is designed to select an intersection first and then adjust its phase split. Two reward functions are meticulously crafted. One focuses on alleviating congestion and the other aims to minimize the total travel time while considering the congestion level. The experiments are carried out with the SUMO traffic simulation software. The performance of the TSC model is evaluated by comparing it with a base case where no signal-timing adjustments are made. The results show that the model can effectively control congestion. For example, the queuing length is significantly reduced in the scenarios tested. Moreover, when the reward is set to both alleviate congestion and minimize the total travel time, the average travel time is remarkably decreased, which indicates that the model can effectively improve traffic conditions. This research provides a new approach for large-scale regional traffic signal control and offers valuable insights for future urban traffic management.

Problem

• Develops a single-agent RL model for regional traffic signal control.
• Aims to reduce traffic congestion and minimize total travel time.
• Evaluates model performance using SUMO traffic simulation software.

Innovation

• Single-agent RL for regional traffic control
• Precise TSC environment with state-action-reward
• SUMO simulation validates congestion reduction