BrainWAM: Action-Space Coordination of Semantic Priors and Predictive Dynamics for Autonomous Driving

2026-08-13

AI Summary

Existing end-to-end autonomous driving approaches struggle to jointly model semantic constraints and dynamic prediction, limiting planning performance. This work proposes a Structured Action Space Coordination framework that encodes semantic priors and dynamic predictions into dedicated action pathways and aligns them at the level of compact action representations. Inspired by neuroscience, the method introduces an action space coordination mechanism to balance attention allocation and employs an asynchronous rectified flow inference strategy to reduce latency. By integrating vision-language-action and world-action models, it achieves efficient inference through decoupled video and action denoising. The approach significantly outperforms baselines relying solely on semantics or prediction, attaining state-of-the-art results on NAVSIM v1 (89.5 PDMS) and v2 (89.6 EPDMS).

Abstract

Autonomous driving requires planning under both semantic constraints and predictive dynamics. Existing end-to-end driving approaches, however, typically emphasize only one side of this requirement: Vision-Language-Action (VLA) models exploit VLM priors for semantic reasoning, while World Action Models (WAMs) provide future-aware prediction through generative world modeling. This naturally motivates a unified planner that can leverage both semantic priors and predictive dynamics. However, we find that a naive combination through joint token-level attention suffers from an attention-allocation mismatch, where semantic shortcuts dominate the shared attention space and suppress predictive dynamics. Inspired by neuroscience evidence that complex behavior arises from coordination among functionally specialized systems, we propose BrainWAM, a structured action-space coordination framework that converts semantic reasoning and predictive world modeling into two specialized action-oriented pathways, and aligns them at the level of compact action representations. We further introduce an asynchronous rectified-flow inference strategy with decoupled video and action denoising, which shortens inference latency while preserving planning-relevant predictive context. BrainWAM reaches state-of-the-art performance on both NAVSIM v1 (89.5 PDMS) and NAVSIM v2 (89.6 EPDMS), consistently outperforming VLA-only or WAM-only methods, highlighting BrainWAM as a practical and promising direction for autonomous driving systems.

Problem

• autonomous driving
• semantic priors
• predictive dynamics
• action-space coordination
• attention allocation

Innovation

• action-space coordination
• semantic priors
• predictive dynamics
• rectified-flow inference
• autonomous driving