A Survey of World Models for Autonomous Driving

2025-01-20

AI Summary

To address insufficient perception-prediction-decision robustness in autonomous driving under long-tail scenarios, multi-source uncertainties, and cross-domain adaptation, this paper systematically surveys advances in world model techniques and proposes, for the first time, a taxonomy grounded in the coupling relationship between “future prediction” and “behavioral planning.” Focusing on four key directions—multi-sensor fusion, 4D occupancy forecasting, generative data synthesis, and self-supervised pretraining—we integrate NeRF-derived representations and spatiotemporal graph neural networks to establish a unified analytical framework covering mainstream methods from 2019–2024. We identify three critical bottlenecks: computational efficiency, simulation fidelity, and full-stack integration. Based on these, we outline an evolutionary roadmap targeting long-tail anomaly detection, domain adaptation, and real-time interactive reasoning—providing theoretical foundations and practical guidelines for next-generation autonomous driving systems that are safe, fair, and efficient.

Technology Category

• Planning, Routing, and Scheduling: Model-Based Reasoning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Recent breakthroughs in autonomous driving have revolutionized the way vehicles perceive and interact with their surroundings. In particular, world models have emerged as a linchpin technology, offering high-fidelity representations of the driving environment that integrate multi-sensor data, semantic cues, and temporal dynamics. Such models unify perception, prediction, and planning, thereby enabling autonomous systems to make rapid, informed decisions under complex and often unpredictable conditions. Research trends span diverse areas, including 4D occupancy prediction and generative data synthesis, all of which bolster scene understanding and trajectory forecasting. Notably, recent works exploit large-scale pretraining and advanced self-supervised learning to scale up models' capacity for rare-event simulation and real-time interaction. In addressing key challenges -- ranging from domain adaptation and long-tail anomaly detection to multimodal fusion -- these world models pave the way for more robust, reliable, and adaptable autonomous driving solutions. This survey systematically reviews the state of the art, categorizing techniques by their focus on future prediction, behavior planning, and the interaction between the two. We also identify potential directions for future research, emphasizing holistic integration, improved computational efficiency, and advanced simulation. Our comprehensive analysis underscores the transformative role of world models in driving next-generation autonomous systems toward safer and more equitable mobility.

Problem

• Autonomous Vehicles
• Environmental Perception
• Sensor Fusion

Innovation

• Autonomous Driving
• Pre-training and Self-supervised Learning
• Model Adaptability and Efficiency