AdvFuzz: Finding More Violations Caused by the EGO Vehicle in Simulation Testing by Adversarial NPC Vehicles

2024-11-29

arXiv.org

AI Summary

In current ADS simulation testing, non-player character (NPC) vehicles employ static, pre-defined behaviors that ignore traffic signals and ego-vehicle interactions, leading to spurious violations caused by unrealistic NPC behavior rather than genuine ego deficiencies—thus impeding root-cause analysis and reducing scenario search efficiency. Method: We propose an adversarial NPC vehicle mechanism embedded in a fuzz-testing framework, integrating reinforcement learning–driven policy optimization, scene constraint modeling, and real-time interactive feedback control to dynamically generate high-interaction scenarios that elicit authentic ego violations. Contribution/Results: Our approach significantly enhances test effectiveness: within 12 hours, it increases violation-scenario discovery by 198.34%; 87.04% of violations are attributable to the ego vehicle—sevenfold higher than baseline; and per-violation localization speed improves by over 92.21%. This work achieves, for the first time, synergistic high attribution accuracy and high efficiency in ADS defect elicitation testing.

Technology Category

• Search and Optimization: Adversarial Search
• Multiagent Systems: Adversarial Agents
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Large-scale security measurements

Abstract

Recently, there has been a significant escalation in both academic and industrial commitment towards the development of autonomous driving systems (ADSs). A number of simulation testing approaches have been proposed to generate diverse driving scenarios for ADS testing. However, scenarios generated by these previous approaches are static and lack interactions between the EGO vehicle and the NPC vehicles, resulting in a large amount of time on average to find violation scenarios. Besides, a large number of the violations they found are caused by aggressive behaviors of NPC vehicles, revealing none bugs of ADS. In this work, we propose the concept of adversarial NPC vehicles and introduce AdvFuzz, a novel simulation testing approach, to generate adversarial scenarios on main lanes (e.g., urban roads and highways). AdvFuzz allows NPC vehicles to dynamically interact with the EGO vehicle and regulates the behaviors of NPC vehicles, finding more violation scenarios caused by the EGO vehicle more quickly. We compare AdvFuzz with a random approach and three state-of-the-art scenario-based testing approaches. Our experiments demonstrate that AdvFuzz can generate 198.34% more violation scenarios compared to the other four approaches in 12 hours and increase the proportion of violations caused by the EGO vehicle to 87.04%, which is more than 7 times that of other approaches. Additionally, AdvFuzz is at least 92.21% faster in finding one violation caused by the EGO vehicle than that of the other approaches.

Problem

• Predefined NPC behaviors ignore traffic signals and Ego vehicle actions
• Unrealistic NPC behaviors produce false violations in ADS testing
• Large scenario search space reduces testing efficiency in prior methods

Innovation

• Dynamic NPC behavior generation during simulation
• Behavior adaptation to traffic signals and Ego vehicle
• Efficient violation scenario discovery for ADS testing