CMU-Drive and V2V-VLA: Cooperative Multi-agent Unified Driving with Reasoning Benchmark and Vehicle-to-Vehicle Vision-Language-Action Models

2026-08-07

AI Summary

This work addresses the limitation of existing end-to-end autonomous driving approaches, which are predominantly confined to single-agent settings and thus unable to support collaborative perception and decision-making among multiple vehicles. To bridge this gap, we introduce CMU-Drive—the first closed-loop, end-to-end cooperative driving benchmark tailored for multi-connected autonomous vehicles—and propose V2V-VLA, a novel model that unifies multi-vehicle visual inputs, language-based reasoning, action generation, and vehicle-to-vehicle (V2V) communication within a unified Vision-Language-Action architecture. V2V-VLA jointly outputs driving actions, trajectory waypoints, natural language explanations, and communication strategies in a single forward pass. Experimental results demonstrate the effectiveness of V2V-VLA on CMU-Drive, establishing a foundational open-source framework—including data, models, and evaluation protocols—for future research in multi-agent cooperative autonomous driving.

Abstract

Vision-Language-Action (VLA) models have recently achieved impressive performance for end-to-end autonomous driving, yet existing approaches are primarily designed for an individual single autonomous driving agent with limited support for cooperative perception, reasoning, and planning. We present Cooperative Multi-agent Unified Driving with Reasoning (CMU-Drive), a closed-loop end-to-end benchmark for evaluating cooperative autonomous driving with multiple connected autonomous vehicles (CAVs) operating in safety-critical driving scenarios with background traffic participants. We further propose Vehicle-to-Vehicle Vision-Language-Action (V2V-VLA), a cooperative VLA model that integrates cooperative driving into a single forward pass by jointly generating driving actions, future waypoints, language reasoning, and communication policies. Experiments on CMU-Drive establish the first benchmark and baseline for cooperative VLA driving and provide a foundation for future research on multi-agent, closed-loop, end-to-end cooperative autonomous driving. Our code, benchmark, and model checkpoint will be publicly released to facilitate open-source research.

Problem

• cooperative autonomous driving
• multi-agent
• Vision-Language-Action models
• connected autonomous vehicles
• closed-loop end-to-end

Innovation

• Cooperative Autonomous Driving
• Vision-Language-Action (VLA)
• Multi-agent Systems
• Vehicle-to-Vehicle Communication
• End-to-End Driving Benchmark