CoInfra: A Large-Scale Cooperative Infrastructure Perception System and Dataset in Adverse Weather

2025-07-02

AI Summary

To address the insufficient robustness of large-scale multi-agent infrastructure perception under adverse weather conditions, this paper proposes a cooperative sensing architecture integrating global time alignment and joint spatial calibration. We introduce the first large-scale, weather-diverse vehicle-infrastructure cooperative dataset—covering rain, fog, and snow—with 195k LiDAR frames and 390k RGB images, each annotated with 3D bounding boxes for five object classes. Technically, the system fuses dual RGB cameras and LiDAR via a latency-aware synchronization protocol and real-time multi-source data fusion, augmented by over-the-air (OTA) management, remote monitoring, and high-definition mapping. Extensive experiments quantify the trade-offs between early and late fusion strategies, demonstrating substantial improvements in detection accuracy and system stability under complex weather. This work establishes a reproducible benchmark platform and delivers key enabling technologies for infrastructure-supported autonomous driving.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Multiagent Systems: Coordination and Collaboration

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Large-scale security measurements

Abstract

We present CoInfra, a large-scale cooperative infrastructure perception system and dataset designed to advance robust multi-agent perception under real-world and adverse weather conditions. The CoInfra system includes 14 fully synchronized sensor nodes, each equipped with dual RGB cameras and a LiDAR, deployed across a shared region and operating continuously to capture all traffic participants in real-time. A robust, delay-aware synchronization protocol and a scalable system architecture that supports real-time data fusion, OTA management, and remote monitoring are provided in this paper. On the other hand, the dataset was collected in different weather scenarios, including sunny, rainy, freezing rain, and heavy snow and includes 195k LiDAR frames and 390k camera images from 8 infrastructure nodes that are globally time-aligned and spatially calibrated. Furthermore, comprehensive 3D bounding box annotations for five object classes (i.e., car, bus, truck, person, and bicycle) are provided in both global and individual node frames, along with high-definition maps for contextual understanding. Baseline experiments demonstrate the trade-offs between early and late fusion strategies, the significant benefits of HD map integration are discussed. By openly releasing our dataset, codebase, and system documentation at https://github.com/NingMingHao/CoInfra, we aim to enable reproducible research and drive progress in infrastructure-supported autonomous driving, particularly in challenging, real-world settings.

Problem

• Advancing robust multi-agent perception in adverse weather
• Providing synchronized sensor data for real-time traffic monitoring
• Enabling infrastructure-supported autonomous driving in challenging conditions

Innovation

• 14 synchronized sensor nodes with dual RGB and LiDAR
• Delay-aware sync protocol and scalable real-time architecture
• 195k LiDAR and 390k camera images in diverse weather