Multi-Objective Reinforcement Learning for Large-Scale Mixed Traffic Control

2025-12-11

AI Summary

Optimizing efficiency, fairness, and safety simultaneously in mixed traffic—comprising human-driven and connected automated vehicles (CAVs) operating under vehicle-infrastructure cooperation—is highly challenging; existing approaches often induce systemic starvation for low-traffic-direction vehicles. To address this, we propose: (1) an active risk-avoidance mechanism based on conflict threat vectors; (2) a queue fairness penalty function to ensure equitable service across all traffic directions; and (3) a hierarchical control framework integrating local multi-objective reinforcement learning with global policy routing. Experiments demonstrate that, compared to baseline methods, our approach reduces average waiting time by 53%, decreases maximum starvation duration by 86%, lowers conflict rate by 86%, and maintains fuel efficiency. Moreover, routing gains scale positively with the penetration rate of autonomous vehicles.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Effective mixed traffic control requires balancing efficiency, fairness, and safety. Existing approaches excel at optimizing efficiency and enforcing safety constraints but lack mechanisms to ensure equitable service, resulting in systematic starvation of vehicles on low-demand approaches. We propose a hierarchical framework combining multi-objective reinforcement learning for local intersection control with strategic routing for network-level coordination. Our approach introduces a Conflict Threat Vector that provides agents with explicit risk signals for proactive conflict avoidance, and a queue parity penalty that ensures equitable service across all traffic streams. Extensive experiments on a real-world network across different robot vehicle (RV) penetration rates demonstrate substantial improvements: up to 53% reductions in average wait time, up to 86% reductions in maximum starvation, and up to 86% reduction in conflict rate compared to baselines, while maintaining fuel efficiency. Our analysis reveals that strategic routing effectiveness scales with RV penetration, becoming increasingly valuable at higher autonomy levels. The results demonstrate that multi-objective optimization through well-curated reward functions paired with strategic RV routing yields significant benefits in fairness and safety metrics critical for equitable mixed-autonomy deployment.

Problem

• Balancing efficiency, fairness, and safety in mixed traffic control
• Addressing systematic starvation of vehicles on low-demand approaches
• Reducing conflicts and ensuring equitable service across traffic streams

Innovation

• Hierarchical framework with multi-objective reinforcement learning
• Conflict Threat Vector for proactive risk avoidance
• Queue parity penalty ensuring equitable traffic service