DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Driving

2026-03-11

AI Summary

Existing autonomous driving reasoning approaches struggle to simultaneously achieve fine-grained modeling of environmental dynamics and efficient decision-making. This work proposes DynVLA, a novel model that introduces the “Dynamics Chain-of-Thought” (Dynamics CoT) paradigm. By decoupling the ego-vehicle’s actions from environmental dynamics, DynVLA first predicts a compact representation of future world states before generating driving commands, thereby enhancing both the physical plausibility and computational efficiency of decisions. The model incorporates dynamic token compression to represent future states concisely and leverages a dynamic tokenizer alongside supervised fine-tuning (SFT) and reinforcement fine-tuning (RFT). Evaluated on NAVSIM, Bench2Drive, and a large-scale internal dataset, DynVLA significantly outperforms existing methods such as Textual CoT and Visual CoT.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Computer Vision: Diffusion Models for Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We propose DynVLA, a driving VLA model that introduces a new CoT paradigm termed Dynamics CoT. DynVLA forecasts compact world dynamics before action generation, enabling more informed and physically grounded decision-making. To obtain compact dynamics representations, DynVLA introduces a Dynamics Tokenizer that compresses future evolution into a small set of dynamics tokens. Considering the rich environment dynamics in interaction-intensive driving scenarios, DynVLA decouples ego-centric and environment-centric dynamics, yielding more accurate world dynamics modeling. We then train DynVLA to generate dynamics tokens before actions through SFT and RFT, improving decision quality while maintaining latency-efficient inference. Compared to Textual CoT, which lacks fine-grained spatiotemporal understanding, and Visual CoT, which introduces substantial redundancy due to dense image prediction, Dynamics CoT captures the evolution of the world in a compact, interpretable, and efficient form. Extensive experiments on NAVSIM, Bench2Drive, and a large-scale in-house dataset demonstrate that DynVLA consistently outperforms Textual CoT and Visual CoT methods, validating the effectiveness and practical value of Dynamics CoT.

Problem

• Autonomous Driving
• World Dynamics
• Action Reasoning
• Chain-of-Thought
• Spatiotemporal Understanding

Innovation

• Dynamics CoT
• Dynamics Tokenizer
• world dynamics modeling
• VLA
• autonomous driving