The Role of World Models in Shaping Autonomous Driving: A Comprehensive Survey

2025-02-14

AI Summary

This survey addresses the challenge of enabling robust, interpretable, and interactive autonomous driving through world modeling. It systematically reviews recent advances in Driving World Models (DWMs), emphasizing their core capability—predicting spatiotemporal evolution of dynamic driving scenes—to enhance perception, scene understanding, and agent-environment interaction. Methodologically, it introduces the first unified taxonomy spanning cross-modal modalities (vision-language-motion) and diverse technical paradigms (NeRF, spatiotemporal diffusion, Transformer-based sequence modeling, and multi-sensor fusion), synthesizing over 100 works. It further curates high-impact datasets and task-specific evaluation metrics, and releases an authoritative open-source literature repository (GitHub Awesome-World-Model). Key limitations identified include scalability, causal reasoning, and real-time inference constraints. Based on this analysis, the survey proposes three concrete future research directions. As the first comprehensive technical roadmap for DWMs in end-to-end autonomous driving, it establishes a foundational framework for paradigm evolution in this rapidly advancing field.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Planning, Routing, and Scheduling: Model-Based Reasoning
• Computer Vision: Vision for Robotics & Autonomous Driving

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Driving World Model (DWM), which focuses on predicting scene evolution during the driving process, has emerged as a promising paradigm in pursuing autonomous driving. These methods enable autonomous driving systems to better perceive, understand, and interact with dynamic driving environments. In this survey, we provide a comprehensive overview of the latest progress in DWM. We categorize existing approaches based on the modalities of the predicted scenes and summarize their specific contributions to autonomous driving. In addition, high-impact datasets and various metrics tailored to different tasks within the scope of DWM research are reviewed. Finally, we discuss the potential limitations of current research and propose future directions. This survey provides valuable insights into the development and application of DWM, fostering its broader adoption in autonomous driving. The relevant papers are collected at https://github.com/LMD0311/Awesome-World-Model.

Problem

• Explores Driving World Models for autonomous driving.
• Categorizes DWM methods by scene prediction modalities.
• Reviews datasets and metrics for DWM research.

Innovation

• Predicts scene evolution autonomously
• Categorizes scene prediction approaches
• Reviews high-impact datasets metrics