RS2AD-LiDAR: End-to-End Autonomous Driving LiDAR Data Generation from Roadside Sensor Observations

2026-05-22

AI Summary

This study addresses the challenges of high data acquisition costs, scarcity of rare scenarios, and data silos in end-to-end autonomous driving by proposing a novel method to reconstruct vehicle-mounted LiDAR point clouds from roadside sensor observations. By unifying coordinate systems, modeling virtual LiDARs, and applying point cloud resampling, the approach efficiently transforms roadside point clouds into a vehicle-centric perspective, yielding high-fidelity synthetic onboard data. This work presents the first cross-view reconstruction from roadside to vehicle-mounted LiDAR and introduces R2V-LiDAR, a dedicated benchmark dataset for evaluation. Experiments demonstrate that the generated data closely matches real vehicle LiDAR in semantic distribution and significantly improves the accuracy of bird’s-eye-view perception and 3D object detection models.

Abstract

End-to-end autonomous driving solutions, which directly process multimodal sensory data and output fine-grained control commands, have gradually become a mainstream direction with the development of autonomous driving technology. However, current methods in this category rely on single-vehicle data collection for model training and optimization, which suffers from high acquisition and annotation costs, scarcity of valuable scenarios, and data silos. To address these challenges, we propose RS2AD-LiDAR, a novel framework for reconstructing and generating vehicle-mounted LiDAR data from roadside sensor observations. Since no public dataset currently provides highly overlapping perception coverage between roadside and vehicle-mounted LiDAR sensors, which is essential for studying roadside-to-vehicle data generation, we constructed a dedicated dataset named R2V-LiDAR which is used solely for evaluation in this work. Specifically, our method transforms roadside LiDAR point clouds into the vehicle-mounted LiDAR coordinate system, and synthesizes high-fidelity vehicle-mounted data via virtual LiDAR modeling and point cloud resampling techniques. To the best of our knowledge, this is the first approach to reconstruct vehicle-mounted LiDAR data from roadside sensor inputs. Extensive experimental comparisons demonstrate the semantic similarity between the generated data and real data. Furthermore, object detection experiments show that incorporating the generated data into real data for model training improves both Bird's Eye View (BEV) and 3D detection accuracy, thereby validating the effectiveness of the proposed method.

Problem

• autonomous driving
• LiDAR data generation
• roadside sensors
• data scarcity
• end-to-end learning

Innovation

• LiDAR data generation
• roadside-to-vehicle reconstruction
• virtual LiDAR modeling
• point cloud resampling
• autonomous driving