End-to-End Autonomous Driving through V2X Cooperation

2024-03-31

arXiv.org

AI Summary

Existing V2X autonomous driving systems primarily enhance isolated functionalities, lacking end-to-end co-optimization across the full perception–mapping–prediction–planning pipeline; moreover, they suffer from inefficient data transmission and fusion under bandwidth constraints. This paper proposes UniV2X, a unified framework featuring a novel sparse-dense hybrid transmission and fusion mechanism, multi-view feature alignment, joint encoding of semantic point clouds and BEV features, a lightweight communication-adaptation module, and a differentiable planning head—enabling holistic end-to-end optimization of the entire V2X stack. Evaluated on DAIR-V2X, UniV2X achieves a 12.7% improvement in planning success rate, with significant gains across all intermediate tasks (e.g., detection, tracking, mapping), while maintaining real-time inference latency. The code is publicly available.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Computer Vision: Multi-modal Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Data transparency and provenance

Abstract

Cooperatively utilizing both ego-vehicle and infrastructure sensor data via V2X communication has emerged as a promising approach for advanced autonomous driving. However, current research mainly focuses on improving individual modules, rather than taking end-to-end learning to optimize final planning performance, resulting in underutilized data potential. In this paper, we introduce UniV2X, a pioneering cooperative autonomous driving framework that seamlessly integrates all key driving modules across diverse views into a unified network. We propose a sparse-dense hybrid data transmission and fusion mechanism for effective vehicle-infrastructure cooperation, offering three advantages: 1) Effective for simultaneously enhancing agent perception, online mapping, and occupancy prediction, ultimately improving planning performance. 2) Transmission-friendly for practical and limited communication conditions. 3) Reliable data fusion with interpretability of this hybrid data. We implement UniV2X, as well as reproducing several benchmark methods, on the challenging DAIR-V2X, the real-world cooperative driving dataset. Experimental results demonstrate the effectiveness of UniV2X in significantly enhancing planning performance, as well as all intermediate output performance. The project is available at href{https://github.com/AIR-THU/UniV2X}{https://github.com/AIR-THU/UniV2X}.

Problem

• V2X Optimization
• Driving Planning
• Data Efficiency

Innovation

• UniV2X
• Data Transmission Optimization
• Unified Driving Framework