Joint-Local Grounded Action Transformation for Sim-to-Real Transfer in Multi-Agent Traffic Control

2025-07-20

AI Summary

To address the significant sim-to-real gap in multi-agent reinforcement learning for traffic signal control (MARL-TSC), this paper introduces Local Grounded Action Transformation (LGAT), the first extension of Grounded Action Transformation (GAT) to decentralized MARL frameworks. LGAT enables robust cooperative control by aggregating local state and action information from neighboring intersections—without requiring global observations. Its key contributions are: (i) the first GAT architecture specifically designed for MARL-TSC; (ii) a fully decentralized design ensuring scalability to large-scale networks; and (iii) localized information fusion that improves generalization under dynamic and adverse conditions. Extensive experiments across diverse road networks and simulated adverse weather scenarios demonstrate that LGAT substantially narrows the sim-to-real gap and consistently outperforms state-of-the-art baselines. Ablation studies validate the effectiveness of each component.

Technology Category

• Multiagent Systems: Multiagent Learning
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Traffic Signal Control (TSC) is essential for managing urban traffic flow and reducing congestion. Reinforcement Learning (RL) offers an adaptive method for TSC by responding to dynamic traffic patterns, with multi-agent RL (MARL) gaining traction as intersections naturally function as coordinated agents. However, due to shifts in environmental dynamics, implementing MARL-based TSC policies in the real world often leads to a significant performance drop, known as the sim-to-real gap. Grounded Action Transformation (GAT) has successfully mitigated this gap in single-agent RL for TSC, but real-world traffic networks, which involve numerous interacting intersections, are better suited to a MARL framework. In this work, we introduce JL-GAT, an application of GAT to MARL-based TSC that balances scalability with enhanced grounding capability by incorporating information from neighboring agents. JL-GAT adopts a decentralized approach to GAT, allowing for the scalability often required in real-world traffic networks while still capturing key interactions between agents. Comprehensive experiments on various road networks under simulated adverse weather conditions, along with ablation studies, demonstrate the effectiveness of JL-GAT. The code is publicly available at https://github.com/DaRL-LibSignal/JL-GAT/.

Problem

• Addressing sim-to-real gap in multi-agent traffic control
• Extending Grounded Action Transformation to MARL frameworks
• Balancing scalability and grounding in decentralized traffic networks

Innovation

• JL-GAT applies GAT to multi-agent RL
• Decentralized GAT enhances scalability and interactions
• Incorporates neighbor info for better grounding