SPHINX: First Explain, Then Explore

2026-06-15

AI Summary

Existing methods for generating adversarial scenarios in autonomous driving rely heavily on large-model priors and struggle to precisely expose vulnerabilities in decision-making policies. This work proposes SPHINX, a closed-loop framework that, for the first time, leverages interpretable evidence derived from the policy itself to guide adversarial scenario generation, adhering to a “explain-then-explore” paradigm. Specifically, it employs explainable AI techniques to identify critical visual concepts responsible for policy failures along with their associated uncertainties, and then synthesizes targeted adversarial scenes using a vision-language model. Evaluated across multiple benchmarks, SPHINX significantly enhances the robustness of mainstream autonomous driving architectures while providing fine-grained, interpretable failure attributions, outperforming existing black-box or non-targeted approaches.

Abstract

Generating adversarial driving scenarios is critical for evaluating and improving autonomous vehicle decision-making systems in simulation. Recent approaches, such as ChatScene and LLM-Attacker, rely primarily on the prior knowledge of Large Language Models and Vision-Language Models to generate driving scenarios procedurally. We argue that adversarial scenes should be generated based on the failure diagnosis (e.g., indecisiveness, multi-frame inconsistency) of the driving policy to specifically address the policy's weaknesses instead of relying on prior assumptions. In this paper, we propose SPHINX, a closed-loop framework for adversarial scenario synthesis guided by a simple principle: first explain, then explore. Beyond blindly exploring the scenario space, SPHINX leverages explainable artificial intelligence methods to analyze the policy, identifying key visual concepts and their influence on policy outputs, and the uncertainty of the decisions. Given the interpretable evidence extracted from the policy's own decision process, we use a vision language model to rationalize and criticize failure modes of the current policy. These critics are then used to generate targeted adversarial scenarios for policy retraining and improvement. We demonstrate that SPHINX can highlight an interpretable account of policy failures while other adversarial scene generation cannot. Across the evaluated benchmarks and test suites, SPHINX can be applied to diverse state-of-the-art autonomous vehicle architectures and yields consistent robustness improvements over existing scenario-generation methods.

Problem

• adversarial driving scenarios
• autonomous vehicle decision-making
• failure diagnosis
• policy robustness
• scenario generation

Innovation

• explainable AI
• adversarial scenario generation
• closed-loop framework
• vision-language model
• policy diagnosis