Collaborative Perception Datasets for Autonomous Driving: A Review

2025-04-17

AI Summary

The field of autonomous driving cooperative perception lacks a systematic survey of available datasets, hindering resource reuse and standardized model evaluation. To address this gap, we present the first comprehensive survey of datasets specifically designed for cooperative perception, introducing a multidimensional classification and evaluation framework covering collaboration paradigms, sensor configurations, data sources, scenario types, and supported tasks. We conduct a cross-dataset comparative analysis of over 30 publicly available datasets. Our analysis identifies six persistent challenges: scalability, diversity, domain adaptability, annotation quality, V2X standardization, and the simulation-to-reality gap. Furthermore, we establish an open-source, continuously updated literature repository and dataset indexing platform (hosted on GitHub), serving as a unified knowledge infrastructure to support algorithm development, benchmarking, and community-driven advancement.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Vision for Robotics & Autonomous Driving
• Multiagent Systems: Coordination and Collaboration

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Abstract

Collaborative perception has attracted growing interest from academia and industry due to its potential to enhance perception accuracy, safety, and robustness in autonomous driving through multi-agent information fusion. With the advancement of Vehicle-to-Everything (V2X) communication, numerous collaborative perception datasets have emerged, varying in cooperation paradigms, sensor configurations, data sources, and application scenarios. However, the absence of systematic summarization and comparative analysis hinders effective resource utilization and standardization of model evaluation. As the first comprehensive review focused on collaborative perception datasets, this work reviews and compares existing resources from a multi-dimensional perspective. We categorize datasets based on cooperation paradigms, examine their data sources and scenarios, and analyze sensor modalities and supported tasks. A detailed comparative analysis is conducted across multiple dimensions. We also outline key challenges and future directions, including dataset scalability, diversity, domain adaptation, standardization, privacy, and the integration of large language models. To support ongoing research, we provide a continuously updated online repository of collaborative perception datasets and related literature: https://github.com/frankwnb/Collaborative-Perception-Datasets-for-Autonomous-Driving.

Problem

• Reviewing collaborative perception datasets for autonomous driving
• Comparing datasets by cooperation paradigms and sensor configurations
• Addressing challenges like scalability and standardization in datasets

Innovation

• Multi-agent information fusion enhances perception
• V2X communication enables diverse datasets
• Systematic review and comparative analysis provided