🤖 AI Summary
To address the high computational cost and poor generalizability of physics-based models in autonomous driving simulation, this paper presents a systematic survey of data-driven camera and LiDAR simulation methods. It introduces, for the first time, a unified taxonomy of sensor simulation paradigms from two complementary perspectives: generative modeling and neural volume rendering. A novel classification framework for volume renderers is proposed based on input encoding types. The survey identifies two critical challenges: the absence of standardized evaluation protocols and limited cross-scenario generalization. Drawing on over 120 scholarly works, it constructs a comprehensive taxonomy, categorizing generative architectures into five types and volume rendering input encodings into four classes; it further synthesizes six mainstream evaluation metrics alongside their applicability boundaries. The work establishes a theoretical foundation and practical guidance for developing efficient, scalable, multimodal sensor simulation models.
📝 Abstract
Perception sensors, particularly camera and Lidar, are key elements of Autonomous Driving Systems (ADS) that enable them to comprehend their surroundings to informed driving and control decisions. Therefore, developing realistic simulation models for these sensors is essential for conducting effective simulation-based testing of ADS. Moreover, the rise of deep learning-based perception models has increased the utility of sensor simulation models for synthesising diverse training datasets. The traditional sensor simulation models rely on computationally expensive physics-based algorithms, specifically in complex systems such as ADS. Hence, the current potential resides in data-driven approaches, fuelled by the exceptional performance of deep generative models in capturing high-dimensional data distribution and volume renderers in accurately representing scenes. This paper reviews the current state-of-the-art data-driven camera and Lidar simulation models and their evaluation methods. It explores a spectrum of models from the novel perspective of generative models and volume renderers. Generative models are discussed in terms of their input-output types, while volume renderers are categorised based on their input encoding. Finally, the paper illustrates commonly used evaluation techniques for assessing sensor simulation models and highlights the existing research gaps in the area.