A Multi-Sensor Dataset for Monitoring the Operational Environment of Rail Vehicles
This work addresses the critical lack of high-quality, large-scale, multi-sensor annotated data tailored to varying levels of railway automation (GoA2–GoA4), which has hindered the safe and efficient deployment of AI-driven systems in rail environments. To bridge this gap, the authors construct and publicly release a multimodal dataset specifically designed for railway operational scenarios, integrating synchronized data from visible-light cameras, LiDAR, and other sensors across diverse operating conditions. Through a rigorous, high-precision manual annotation pipeline, the dataset comprises over 7 million labeled instances, enabling robust detection and classification of both rail-specific and general objects. This resource fills a significant void in training and validation data for fully automated rail systems, substantially advancing the development, evaluation, and real-world deployment of AI models in railway applications.