Getting SMARTER for Motion Planning in Autonomous Driving Systems

2025-02-20

AI Summary

Evaluating autonomous driving motion planners on real vehicles is costly and hazardous, necessitating high-fidelity, customizable simulation platforms and rigorous, reproducible benchmarks. To address this, we introduce SMARTS 2.0—the first open-source simulator supporting real-world map integration, vehicle-to-vehicle (V2V) communication, joint multi-agent traffic–pedestrian modeling, and modular sensor simulation. We further propose the first reproducible benchmark specifically designed for complex interactive scenarios (e.g., uncontrolled intersection negotiation and intent-ambiguous car-following), featuring a quantitative metric suite balancing task-specificity and generalizability. SMARTS 2.0 integrates high-fidelity vehicle dynamics, behavior-cloned traffic models, and a real-time V2V protocol stack. Extensive experiments demonstrate its effectiveness in exposing critical deficiencies of state-of-the-art planners—particularly in dynamic strategic interaction and long-horizon intent reasoning. The platform and benchmark are publicly released and have been widely adopted by the research community.

Technology Category

• Intelligent Robots: Motion and Path Planning
• Multiagent Systems: Multiagent Planning
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Motion planning is a fundamental problem in autonomous driving and perhaps the most challenging to comprehensively evaluate because of the associated risks and expenses of real-world deployment. Therefore, simulations play an important role in efficient development of planning algorithms. To be effective, simulations must be accurate and realistic, both in terms of dynamics and behavior modeling, and also highly customizable in order to accommodate a broad spectrum of research frameworks. In this paper, we introduce SMARTS 2.0, the second generation of our motion planning simulator which, in addition to being highly optimized for large-scale simulation, provides many new features, such as realistic map integration, vehicle-to-vehicle (V2V) communication, traffic and pedestrian simulation, and a broad variety of sensor models. Moreover, we present a novel benchmark suite for evaluating planning algorithms in various highly challenging scenarios, including interactive driving, such as turning at intersections, and adaptive driving, in which the task is to closely follow a lead vehicle without any explicit knowledge of its intention. Each scenario is characterized by a variety of traffic patterns and road structures. We further propose a series of common and task-specific metrics to effectively evaluate the performance of the planning algorithms. At the end, we evaluate common motion planning algorithms using the proposed benchmark and highlight the challenges the proposed scenarios impose. The new SMARTS 2.0 features and the benchmark are publicly available at github.com/huawei-noah/SMARTS.

Problem

• Enhances motion planning simulation accuracy and realism.
• Introduces SMARTS 2.0 for large-scale autonomous driving simulations.
• Proposes a benchmark for evaluating planning algorithms in complex scenarios.

Innovation

• Enhanced simulation for motion planning
• Realistic map and V2V integration
• Novel benchmark for challenging scenarios