Z-Merge: Multi-Agent Reinforcement Learning for On-Ramp Merging with Zone-Specific V2X Traffic Information

2025-11-18

AI Summary

In mixed traffic, autonomous vehicles (AVs) face safety and efficiency challenges during on-ramp merging due to reliance solely on local perception. To address this, we propose a vehicle-infrastructure cooperative multi-agent reinforcement learning framework. Our approach innovatively fuses onboard local sensing with region-level V2X global information provided by roadside units (RSUs), formulating a partially observable Markov decision process (POMDP) and designing a hybrid action space encompassing both lane-changing and longitudinal control. We employ a parameterized deep Q-network and conduct end-to-end training and evaluation on the SUMO-MOSAIC co-simulation platform. Experiments demonstrate significant improvements in merge success rate, traffic throughput, and collision avoidance performance, along with strong generalization across diverse traffic scenarios. This work establishes a scalable, infrastructure-augmented paradigm for autonomous ramp merging.

Technology Category

• Multiagent Systems: Multiagent Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Mixed Discrete/Continuous Planning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Ramp merging is a critical and challenging task for autonomous vehicles (AVs), particularly in mixed traffic environments with human-driven vehicles (HVs). Existing approaches typically rely on either lane-changing or inter-vehicle gap creation strategies based solely on local or neighboring information, often leading to suboptimal performance in terms of safety and traffic efficiency. In this paper, we present a V2X (vehicle-to-everything communication)-assisted Multiagent Reinforcement Learning (MARL) framework for on-ramp merging that effectively coordinates the complex interplay between lane-changing and inter-vehicle gap adaptation strategies by utilizing zone-specific global information available from a roadside unit (RSU). The merging control problem is formulated as a Multiagent Partially Observable Markov Decision Process (MA-POMDP), where agents leverage both local and global observations through V2X communication. To support both discrete and continuous control decisions, we design a hybrid action space and adopt a parameterized deep Q-learning approach. Extensive simulations, integrating the SUMO traffic simulator and the MOSAIC V2X simulator, demonstrate that our framework significantly improves merging success rate, traffic efficiency, and road safety across diverse traffic scenarios.

Problem

• Improves autonomous vehicle on-ramp merging in mixed traffic
• Coordinates lane-changing and gap adaptation using V2X communication
• Enhances merging success rate, traffic efficiency, and road safety

Innovation

• V2X-assisted MARL framework for ramp merging
• Hybrid action space with parameterized Q-learning
• Zone-specific global information via roadside units