WHALES: A Multi-agent Scheduling Dataset for Enhanced Cooperation in Autonomous Driving

2024-11-20

arXiv.org

AI Summary

Existing cooperative perception research is hindered by the lack of datasets capable of modeling realistic V2X interactions—particularly under dynamic communication constraints. To address this, we introduce WHALES, the first large-scale multi-agent scheduling dataset specifically designed for vehicle-to-vehicle (V2V) cooperative perception, featuring an average of 8.4 collaborating agents per sequence and thus overcoming agent-scale limitations. We further propose a novel task paradigm: dynamic agent scheduling optimized for perceptual gain maximization. Leveraging CARLA, we generate high-fidelity driving sequences, simulate multi-view sensors, and annotate agent behaviors. Experiments demonstrate that our approach improves mAP for object detection under occlusion by 12.3%. Both the dataset and implementation code are publicly released, establishing a new benchmark and reproducible foundation for multi-agent cooperative perception research.

Technology Category

• Multiagent Systems: Teamwork
• Planning, Routing, and Scheduling: Model-Based Reasoning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Achieving high levels of safety and reliability in autonomous driving remains a critical challenge, especially due to occlusion and limited perception ranges in standalone systems. Cooperative perception among vehicles offers a promising solution, but existing research is hindered by datasets with a limited number of agents. Scaling up the number of cooperating agents is non-trivial and introduces significant computational and technical hurdles that have not been addressed in previous works. To bridge this gap, we present Wireless enHanced Autonomous vehicles with Large number of Engaged agentS (WHALES), a dataset generated using CARLA simulator that features an unprecedented average of 8.4 agents per driving sequence. In addition to providing the largest number of agents and viewpoints among autonomous driving datasets, WHALES records agent behaviors, enabling cooperation across multiple tasks. This expansion allows for new supporting tasks in cooperative perception. As a demonstration, we conduct experiments on agent scheduling task, where the ego agent selects one of multiple candidate agents to cooperate with, optimizing perception gains in autonomous driving. The WHALES dataset and codebase can be found at https://github.com/chensiweiTHU/WHALES.

Problem

• Lack of datasets for real-world V2X interaction complexity
• Need for communication-aware agent scheduling benchmarks
• Bridging simulation-real gap in cooperative perception research

Innovation

• First large-scale V2X dataset for cooperative perception
• Integrates communication metadata for realistic bottlenecks
• Proposes Coverage-Aware Historical Scheduler (CAHS)