Foundation Models for Rapid Autonomy Validation

2024-10-22

arXiv.org

AI Summary

Autonomous driving safety verification faces two key challenges: insufficient coverage of rare hazardous scenarios and low efficiency of traditional million-mile testing. To address these, this paper proposes the first behavior-centric foundation model for verification. It employs a driving-scene Masked Autoencoder (MAE) to learn robust scenario embeddings, then integrates embedding-space clustering with collision-difficulty prediction to establish a dual importance-weighted sampling mechanism—ensuring both broad scenario diversity and focused exploration of high-risk subspaces. The method significantly improves exposure efficiency for rare collision events, yielding more accurate and lower-variance estimates of collision rate and severity at equivalent simulation mileage. This approach establishes a scalable, high-confidence paradigm for safety verification of autonomous driving systems.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Natural Language Processing: Safety and Robustness
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

We are motivated by the problem of autonomous vehicle performance validation. A key challenge is that an autonomous vehicle requires testing in every kind of driving scenario it could encounter, including rare events, to provide a strong case for safety and show there is no edge-case pathological behavior. Autonomous vehicle companies rely on potentially millions of miles driven in realistic simulation to expose the driving stack to enough miles to estimate rates and severity of collisions. To address scalability and coverage, we propose the use of a behavior foundation model, specifically a masked autoencoder (MAE), trained to reconstruct driving scenarios. We leverage the foundation model in two complementary ways: we (i) use the learned embedding space to group qualitatively similar scenarios together and (ii) fine-tune the model to label scenario difficulty based on the likelihood of a collision upon simulation. We use the difficulty scoring as importance weighting for the groups of scenarios. The result is an approach which can more rapidly estimate the rates and severity of collisions by prioritizing hard scenarios while ensuring exposure to every kind of driving scenario.

Problem

• Validating autonomous vehicle performance in diverse driving scenarios
• Addressing scalability and coverage in collision rate estimation
• Prioritizing hard scenarios for rapid collision severity assessment

Innovation

• Uses masked autoencoder for scenario reconstruction
• Groups similar scenarios via embedding space
• Fine-tunes model to label collision likelihood