Analyzing the Impact of Simulation Fidelity on the Evaluation of Autonomous Driving Motion Control

2024-06-02

2024 IEEE Intelligent Vehicles Symposium (IV)

AI Summary

This study addresses the challenge of inconsistent evaluation of autonomous driving control algorithms due to varying fidelity levels in vehicle dynamics models. The authors develop a high-fidelity vehicle model compatible with Autoware and systematically derive a hierarchy of reduced-fidelity models through controlled simplifications. Leveraging over 550 simulations and real-world track data—encompassing speeds up to 267 kph and lateral accelerations of 15 m/s²—they quantitatively assess, for the first time, how simulation fidelity impacts trajectory tracking performance. Furthermore, they propose an application-oriented model simplification criterion based on acceleration margin, which explicitly defines acceptable levels of model reduction under different acceleration limits. This framework provides a principled basis for selecting appropriate model fidelity when evaluating control algorithms in simulation.

Technology Category

• Planning, Routing, and Scheduling: Model-Based Reasoning
• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Simulation is crucial in the development of autonomous driving software. In particular, assessing control algorithms requires an accurate vehicle dynamics simulation. However, recent publications use models with varying levels of detail. This disparity makes it difficult to compare individual control algorithms. Therefore, this paper aims to investigate the influence of the fidelity of vehicle dynamics modeling on the closed-loop behavior of trajectory-following controllers. For this purpose, we introduce a comprehensive Autoware-compatible vehicle model. By simplifying this, we derive models with varying fidelity. Evaluating over 550 simulation runs allows us to quantify each model’s approximation quality compared to real-world data. Furthermore, we investigate whether the influence of model simplifications changes with varying margins to the acceleration limit of the vehicle. From this, we deduce to which degree a vehicle model can be simplified to evaluate control algorithms depending on the specific application. The real-world data used to validate the simulation environment originate from the Indy Autonomous Challenge race at the Autodromo Nazionale di Monza in June 2023. They show the fastest fully autonomous lap of TUM Autonomous Motorsport, with vehicle speeds reaching $267\frac{{{\text{km}}}}{{\text{h}}}$ and lateral accelerations of up to $15\frac{{{\text{mm}}}}{{{{\text{s}}^2}}}$.

Problem

• simulation fidelity
• autonomous driving
• vehicle dynamics
• motion control
• trajectory-following controllers

Innovation

• simulation fidelity
• vehicle dynamics modeling
• autonomous driving control
• trajectory-following controller
• Autoware-compatible simulation