Simulation-Based Performance Evaluation of 3D Object Detection Methods with Deep Learning for a LiDAR Point Cloud Dataset in a SOTIF-related Use Case
Current SOTIF (Safety of the Intended Functionality) validation for autonomous driving lacks systematic assessment of LiDAR perception robustness under adverse weather conditions. Method: This work formally defines and models SOTIF-critical use cases, and constructs a high-fidelity simulated LiDAR point cloud dataset—comprising 547 frames across 21 weather-illumination combinations—using CARLA and PreScan. Within the MMDetection3D and OpenPCDet frameworks, it conducts a comprehensive performance evaluation of state-of-the-art 3D object detectors using AP and Recall metrics for cross-model comparison. Contribution/Results: Results reveal significant performance degradation under rain, fog, and low-light conditions, with AP dropping by up to 62%. To our knowledge, this is the first study to provide a reproducible, quantitative benchmark for evaluating LiDAR perception robustness in SOTIF contexts. The dataset and empirical findings bridge a critical gap in SOTIF validation, supporting algorithmic refinement and standardization efforts.