Self-driving cars: Are we there yet?
🤖 AI Summary
Autonomous motion planning faces integrated challenges in dynamic environment perception, multi-agent trajectory prediction, and safe, efficient decision-making; however, heterogeneous evaluation platforms (e.g., CARLA, nuPlan, Waymo) employ inconsistent benchmarks, hindering fair cross-algorithm comparison. This work presents the first cross-platform benchmark evaluation of leading open-benchmark motion planners—drawn from three major public leaderboards—within a unified simulation environment (CARLA v2.0). Leveraging standardized API adaptation and comprehensive testing across diverse scenarios (e.g., traffic congestion, uncontrolled intersections, dense pedestrian zones), we systematically quantify disparities and common bottlenecks in prediction accuracy, planning robustness, and real-time performance. We introduce the first reproducible, cross-platform evaluation framework, empirically identifying critical limitations—including inadequate long-horizon interaction modeling and weak coordination with heterogeneous traffic participants. Our findings provide evidence-based guidance for algorithmic advancement and standardization of autonomous driving evaluation protocols.
Application Category
📝 Abstract
Autonomous driving remains a highly active research domain that seeks to enable vehicles to perceive dynamic environments, predict the future trajectories of traffic agents such as vehicles, pedestrians, and cyclists and plan safe and efficient future motions. To advance the field, several competitive platforms and benchmarks have been established to provide standardized datasets and evaluation protocols. Among these, leaderboards by the CARLA organization and nuPlan and the Waymo Open Dataset have become leading benchmarks for assessing motion planning algorithms. Each offers a unique dataset and challenging planning problems spanning a wide range of driving scenarios and conditions. In this study, we present a comprehensive comparative analysis of the motion planning methods featured on these three leaderboards. To ensure a fair and unified evaluation, we adopt CARLA leaderboard v2.0 as our common evaluation platform and modify the selected models for compatibility. By highlighting the strengths and weaknesses of current approaches, we identify prevailing trends, common challenges, and suggest potential directions for advancing motion planning research.
Problem
Research questions and friction points this paper is trying to address.
Analyzing motion planning methods for autonomous vehicles in dynamic environments
Comparing performance across CARLA, nuPlan, and Waymo Open Dataset benchmarks
Identifying challenges and future directions for autonomous driving research
Innovation
Methods, ideas, or system contributions that make the work stand out.
Comparative analysis of motion planning leaderboards
Adopting CARLA platform for unified evaluation
Modifying models for cross-platform compatibility assessment
🔎 Similar Papers
No similar papers found.
