Conformal Trajectory Prediction with Multi-View Data Integration in Cooperative Driving

2024-08-01

arXiv.org

AI Summary

To address low trajectory prediction accuracy caused by limited sensor field-of-view and occlusions in single-vehicle perception, this paper proposes V2INet—the first end-to-end multi-view trajectory prediction framework designed for V2X cooperative driving. Methodologically, it fuses heterogeneous V2V and V2I observations, employs a Transformer for cross-view feature alignment, leverages graph neural networks to model inter-vehicle and infrastructure-vehicle interactions, and introduces a novel posterior conformal prediction module to generate statistically valid confidence regions—marking the first application of conformal prediction in trajectory forecasting. Evaluated on the V2X-Seq dataset, V2INet achieves a 19.3% reduction in Final Displacement Error (FDE) and a 26.7% decrease in Miss Rate (MR), while maintaining real-time inference on a single GPU. The source code is publicly available.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Current research on trajectory prediction primarily relies on data collected by onboard sensors of an ego vehicle. With the rapid advancement in connected technologies, such as vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication, valuable information from alternate views becomes accessible via wireless networks. The integration of information from alternative views has the potential to overcome the inherent limitations associated with a single viewpoint, such as occlusions and limited field of view. In this work, we introduce V2INet, a novel trajectory prediction framework designed to model multi-view data by extending existing single-view models. Unlike previous approaches where the multi-view data is manually fused or formulated as a separate training stage, our model supports end-to-end training, enhancing both flexibility and performance. Moreover, the predicted multimodal trajectories are calibrated by a post-hoc conformal prediction module to get valid and efficient confidence regions. We evaluated the entire framework using the real-world V2I dataset V2X-Seq. Our results demonstrate superior performance in terms of Final Displacement Error (FDE) and Miss Rate (MR) using a single GPU. The code is publicly available at: url{https://github.com/xichennn/V2I_trajectory_prediction}.

Problem

• Integrates multi-view data for trajectory prediction
• Overcomes single-view limitations like occlusions
• Provides calibrated confidence regions for predictions

Innovation

• Multi-view data integration via V2V and V2I communication
• End-to-end training for enhanced flexibility and performance
• Conformal prediction for calibrated confidence regions