World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning

2026-07-12

AI Summary

This work addresses the lack of robust planning capabilities in autonomous driving under dense traffic, particularly for rare safety-critical scenarios. It proposes Adversarial World Modeling (AWM), a framework that formalizes robust planning as a constrained minimax game. Through multi-agent self-play fine-tuning, AWM jointly optimizes an autoregressive planner and a role-conditioned adversary, uniquely transforming the world model into a learnable, sparse, and scene-adaptive adversarial coalition. The approach incorporates a decoupled solver, a regret-aware response mechanism, and a tail-risk-weighted policy to significantly enhance recovery performance in extreme scenarios while preserving standard driving capabilities. Experiments demonstrate that AWM generates transferable adversarial interactions on the nuPlan and InterPlan benchmarks, effectively improving both robustness and performance of closed-loop systems in both common and long-tail, highly interactive traffic situations.

Abstract

Robust motion planning in dense traffic requires autonomous vehicles to interact in rare and safety-critical scenarios that are underrepresented in naturalistic driving data. Although adversarial training offers a feasible solution, existing methods often rely on external scenario generators, heuristic perturbations, or simulator-heavy rollouts, which makes them difficult to integrate with modern autoregressive planners. Here, we cast adversarially robust planner learning as a constrained min-max game and propose Adversarial World Modeling (AWM), a theoretically grounded multi-agent self-play fine-tuning framework. Since solving the exact game is intractable, AWM introduces a principled decoupled solver. In the inner minimization, the planner's predictive world model is converted into a role-conditioned adversary that learns sparse, scene-adaptive attack coalitions via counterfactual credit assignment. In the outer maximization, the ego planner optimizes a regret-aware robust best response against the frozen AWM, utilizing tail-risk weighting and reference-anchored trust regions to improve hard-case recovery while preserving nominal driving behavior. Experiments on the nuPlan and InterPlan benchmarks demonstrate that our method generates transferable adversarial interactions and yields a robust planner that achieves competitive closed-loop performance in both nominal and highly interactive long-tail scenarios. Theoretical analysis justifies the decoupled solver and the main optimization components.

Problem

• robust motion planning
• adversarial training
• rare scenarios
• safety-critical interactions
• autonomous driving

Innovation

• Adversarial World Modeling
• multi-agent self-play
• robust motion planning
• counterfactual credit assignment
• tail-risk weighting