From Attacks to Curricula: Learnability-Guided Adversarial Training for Safe Autonomous Driving

2026-06-11

AI Summary

This work addresses the limitations of conventional adversarial training in autonomous driving, which relies on attack-oriented scenario generation and often produces unsolvable extreme cases while neglecting the evolution of policy capabilities, leading to low learning efficiency and slow convergence. To overcome these issues, the authors propose AlignADV, a novel framework that shifts adversarial training from an attack-centric paradigm to a learnability-oriented one. AlignADV leverages direct preference optimization to generate critical yet solvable scenarios aligned with the agent’s current policy capabilities, dynamically predicted via behavioral fingerprints and multimodal models. A dynamic curriculum sampling mechanism further enables closed-loop adversarial training. Evaluated on the Waymo dataset, AlignADV reduces training steps by 40.6%, significantly lowers collision rates, and improves route completion rates, demonstrating superior performance under both normal and adversarial traffic conditions.

Abstract

Closed-loop adversarial training improves autonomous driving safety by exposing policies to rare safety-critical scenarios. Standard pipelines first generate adversarial scenarios and then sample them for policy optimization. However, most existing frameworks remain attack-oriented: collision-driven generators often synthesize unsolvable extreme situations, which can degrade learning, while heuristic samplers ignore the evolving capability of the driving policy, causing sample inefficiency and delayed convergence. We propose AlignADV, a learnability-guided closed-loop adversarial training framework that converts adversarial scenarios into resolvable and capability-aligned curricula. First, we reformulate adversarial scenario generation as a preference alignment problem and employ direct preference optimization to guide the generator toward critical yet resolvable scenarios. Second, we introduce behavioral fingerprints to capture the intrinsic characteristics of the evolving policy and construct a multi-modal capability prediction model that estimates policy performance without expensive closed-loop simulations. By combining resolvability-aligned scenarios with capability predictions, AlignADV develops a dynamic curriculum sampling mechanism that prioritizes scenarios targeting the current policy's vulnerabilities. Experiments on the Waymo Open Motion Dataset demonstrate that AlignADV improves convergence efficiency and final performance, reducing training steps by up to 40.6 percent compared with baseline methods while lowering collision rate and improving route completion under both normal and adversarial traffic conditions. These results highlight a shift from attack-oriented scenario generation to learnability-guided policy improvement, offering a principled direction for safer and more efficient autonomous driving training. Project page: https://meiyuewen.github.io/AlignADV/.

Problem

• adversarial training
• autonomous driving
• scenario generation
• curriculum learning
• policy optimization

Innovation

• learnability-guided training
• adversarial scenario generation
• preference alignment
• behavioral fingerprints
• curriculum sampling