VLMLight: Traffic Signal Control via Vision-Language Meta-Control and Dual-Branch Reasoning

2025-05-26

AI Summary

Addressing the challenge of simultaneously ensuring real-time responsiveness, safety, and interpretability in urban intersection traffic signal control, this paper proposes a safety-first dual-path decision framework. One path employs multi-view visual perception and real-time traffic simulation to dynamically model traffic states; the other integrates a large language model (LLM)-driven safety meta-controller for natural-language rule guidance and structured collaborative reasoning. We introduce the first image-based traffic simulator to support high-fidelity training and validation. Our method reduces average emergency vehicle waiting time by 65% while incurring less than 1% degradation in baseline traffic throughput. The core contribution is a novel cross-modal meta-control paradigm unifying vision, language, and reinforcement learning—delivering provable safety guarantees, millisecond-level response latency, and human-interpretable, rule-grounded decision logic.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Computer Vision: Language and Vision
• Natural Language Processing: Safety and Robustness

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Traffic signal control (TSC) is a core challenge in urban mobility, where real-time decisions must balance efficiency and safety. Existing methods - ranging from rule-based heuristics to reinforcement learning (RL) - often struggle to generalize to complex, dynamic, and safety-critical scenarios. We introduce VLMLight, a novel TSC framework that integrates vision-language meta-control with dual-branch reasoning. At the core of VLMLight is the first image-based traffic simulator that enables multi-view visual perception at intersections, allowing policies to reason over rich cues such as vehicle type, motion, and spatial density. A large language model (LLM) serves as a safety-prioritized meta-controller, selecting between a fast RL policy for routine traffic and a structured reasoning branch for critical cases. In the latter, multiple LLM agents collaborate to assess traffic phases, prioritize emergency vehicles, and verify rule compliance. Experiments show that VLMLight reduces waiting times for emergency vehicles by up to 65% over RL-only systems, while preserving real-time performance in standard conditions with less than 1% degradation. VLMLight offers a scalable, interpretable, and safety-aware solution for next-generation traffic signal control.

Problem

• Balancing efficiency and safety in traffic signal control
• Generalizing to complex, dynamic, and safety-critical traffic scenarios
• Integrating vision-language meta-control with dual-branch reasoning

Innovation

• Vision-language meta-control for traffic signals
• Dual-branch reasoning for safety and efficiency
• Image-based simulator with multi-view perception