Zero-Shot MARL Benchmark in the Cyber-Physical Mobility Lab

2026-01-23

AI Summary

This work addresses the lack of reproducible benchmarks for systematically evaluating zero-shot sim-to-real transfer of multi-agent reinforcement learning (MARL) policies in connected autonomous driving. To bridge this gap, the authors establish a unified benchmark that integrates high-fidelity digital twins, a physical testbed, and simulation environments within the Cyber-Physical Mobility Lab framework. For the first time, this setup enables end-to-end, zero-shot deployment and evaluation of MARL policies across all three domains under rigorously reproducible real-world conditions. By deploying the SigmaRL policy, the study quantitatively reveals how discrepancies in control architectures and environmental fidelity critically contribute to performance degradation during transfer. The resulting framework provides an open-source, structured, and reproducible foundation for advancing MARL research in sim-to-real transfer.

Technology Category

• Multiagent Systems: Multiagent Learning
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Multi-Robot Systems

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We present a reproducible benchmark for evaluating sim-to-real transfer of Multi-Agent Reinforcement Learning (MARL) policies for Connected and Automated Vehicles (CAVs). The platform, based on the Cyber-Physical Mobility Lab (CPM Lab) [1], integrates simulation, a high-fidelity digital twin, and a physical testbed, enabling structured zero-shot evaluation of MARL motion-planning policies. We demonstrate its use by deploying a SigmaRL-trained policy [2] across all three domains, revealing two complementary sources of performance degradation: architectural differences between simulation and hardware control stacks, and the sim-to-real gap induced by increasing environmental realism. The open-source setup enables systematic analysis of sim-to-real challenges in MARL under realistic, reproducible conditions.

Problem

• Zero-Shot
• Sim-to-Real Transfer
• Multi-Agent Reinforcement Learning
• Connected and Automated Vehicles
• Benchmark

Innovation

• Zero-Shot MARL
• Sim-to-Real Transfer
• Cyber-Physical Mobility Lab
• Digital Twin
• Connected and Automated Vehicles