RMP-YOLO: A Robust Motion Predictor for Partially Observable Scenarios even if You Only Look Once

2024-09-18

arXiv.org

AI Summary

To address the insufficient robustness of local-observable motion prediction in autonomous driving caused by missing or noisy historical trajectories, this paper proposes a “reconstruction-first” unified framework. First, a scene tokenization module fuses spatiotemporal and map-topological features; second, a graph neural network jointly models multi-agent interactions to achieve noise-robust trajectory reconstruction; finally, predictions are generated end-to-end. Key contributions include: (1) the first reconstruction-first paradigm for motion prediction; (2) a plug-and-play trajectory recovery module that explicitly encodes road geometry and proximal agent interactions; and (3) a YOLO-style lightweight prediction architecture. Evaluated on the 2024 Waymo Motion Prediction Challenge, the method ranks third overall and demonstrates significant accuracy improvements under varying degrees of trajectory occlusion and observation noise. The framework has been deployed in production vehicle systems.

Technology Category

• Computer Vision: Motion & Tracking
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We introduce RMP-YOLO, a unified framework designed to provide robust motion predictions even with incomplete input data. Our key insight stems from the observation that complete and reliable historical trajectory data plays a pivotal role in ensuring accurate motion prediction. Therefore, we propose a new paradigm that prioritizes the reconstruction of intact historical trajectories before feeding them into the prediction modules. Our approach introduces a novel scene tokenization module to enhance the extraction and fusion of spatial and temporal features. Following this, our proposed recovery module reconstructs agents' incomplete historical trajectories by leveraging local map topology and interactions with nearby agents. The reconstructed, clean historical data is then integrated into the downstream prediction modules. Our framework is able to effectively handle missing data of varying lengths and remains robust against observation noise, while maintaining high prediction accuracy. Furthermore, our recovery module is compatible with existing prediction models, ensuring seamless integration. Extensive experiments validate the effectiveness of our approach, and deployment in real-world autonomous vehicles confirms its practical utility. In the 2024 Waymo Motion Prediction Competition, our method, RMP-YOLO, achieves state-of-the-art performance, securing third place.

Problem

• Predicts motion robustly with incomplete input data
• Reconstructs historical trajectories for accurate prediction
• Handles missing data and observation noise effectively

Innovation

• Reconstructs incomplete historical trajectories first
• Uses scene tokenization for spatiotemporal feature fusion
• Integrates recovery module with existing prediction models