Explainable Reinforcement Learning for Adaptive Traffic Signal Control

2026-07-04

AI Summary

This work addresses the limited interpretability of deep reinforcement learning in adaptive traffic signal control, which hinders its deployment in safety-critical scenarios. The authors propose an entity-centric, interpretable reinforcement learning framework that models intersection states as lane entities structured by phase timing. By integrating a two-stage attention mechanism—comprising multi-head cross-attention and self-attention—the approach captures inter-lane dependencies while incorporating action masking to enforce safe and compliant signal control. This is the first method to combine structured entity representations with attention mechanisms to enable visualizable decision-making. Experimental results in microscopic traffic simulation demonstrate significant performance gains over existing approaches, substantially reducing vehicle delay. Moreover, the learned attention weights align closely with established traffic engineering principles, thereby enhancing the system’s auditability and trustworthiness.

Abstract

Reinforcement Learning (RL) has emerged as a powerful paradigm for adaptive traffic signal control. However, in safety-critical infrastructure like traffic control, the opaque, black-box nature of deep RL models poses challenges for transportation agency acceptance, regulatory compliance, operational trust, troubleshooting, and fine-tuning. To bridge this gap between high-performance optimization and human-comprehensible interpretability, this effort introduces a novel, explainable entity centric RL framework for safe and transparent traffic signal control. Rather than processing traffic states through monolithic, flat vectors, the proposed architecture disaggregates real-time intersection observations into distinct, high-dimensional lane entities and phase temporal configurations to inherently preserve the structural topology and geometric configurations of the intersection. Relational dependencies and inter-lane conflicts are dynamically extracted via a dual-stage attention network featuring sequential multi-head cross-attention and self-attention blocks. This design yields a real time affinity matrix that quantifies the direct influence of signal phases on specific approach volumes and queues, providing full visual and analytical interpretability. To ensure strict operational reliability, a deterministic action-masking interface is integrated directly into the Proximal Policy Optimization pipeline, explicitly blocking invalid phase transitions to guarantee absolute compliance with established signal timing and safety constraints. Evaluated in a microscopic simulation environment, outperforms state-of-the-art baselines in delay minimization. More importantly, the emergent attention weights align precisely with established traffic engineering principles, offering an auditable, trust-enabling, and deployable architecture for next-generation adaptive traffic control systems.

Problem

• Explainable Reinforcement Learning
• Adaptive Traffic Signal Control
• Interpretability
• Safety-Critical Systems
• Traffic Signal Optimization

Innovation

• Explainable Reinforcement Learning
• Entity-Centric Representation
• Dual-Stage Attention Network
• Action Masking
• Adaptive Traffic Signal Control