4DLidarOpen: An Open 4D FMCW Lidar Dataset for Motion-Aware Autonomous Driving

2026-05-18

AI Summary

Existing LiDAR datasets commonly lack point-wise radial velocity information, limiting accurate motion perception in dynamic traffic scenarios. To address this gap, this work presents a large-scale multimodal autonomous driving dataset that, for the first time, publicly provides urban driving data captured with 4D FMCW LiDAR featuring point-level radial velocities. The dataset integrates multiple LiDAR modalities, surround-view cameras, and vehicle poses, delivering velocity-annotated point clouds along with 3D object annotations equipped with trajectory IDs. By jointly leveraging geometric and velocity representations, the dataset substantially enhances motion perception capabilities—particularly for vulnerable road users and high-speed objects—and demonstrates consistent performance gains across multiple tasks, including BEV segmentation, scene flow prediction, 3D detection, and motion planning, with especially notable improvements in complex urban environments.

Abstract

We present 4DLidarOpen, a large-scale open multi-modal dataset for autonomous driving, centered on 4D frequency-modulated continuous-wave (FMCW) Lidar sensing. Unlike conventional time-of-flight Lidar datasets that mainly provide geometric measurements, 4DLidarOpen includes point-wise radial velocity measurements from a forward-facing 4D FMCW Lidar, together with multiple Lidars of different types, including rotating, solid-state, and blind-spot variants, surround-view cameras, and 6-DOF ego-vehicle poses. The dataset was collected in complex urban environments in Beijing and covers dense pedestrian interactions, congested traffic, high-speed driving, and unprotected maneuvers. 4DLidarOpen provides synchronized multi-sensor data and 3D bounding-box annotations with persistent track IDs across five object categories. A hybrid annotation strategy is adopted, where large-scale auto-labeled data support scalable training and human experts refine annotations for the human-annotated training and validation sets. Based on this dataset, we establish benchmarks for 3D object detection, birds-eye view (BEV) segmentation and flow prediction, and motion forecasting with planning. Extensive experiments show that direct velocity measurements from 4D FMCW Lidar provide complementary motion cues for dynamic-scene understanding. Compared with geometric-only sensing, the velocity-aware representation improves motion-related perception and downstream forecasting and planning, especially in scenarios involving vulnerable road users and fast-moving objects. These results indicate that 4D FMCW Lidar is a promising sensing modality for motion-aware autonomous driving. The dataset and evaluation toolkit are publicly released to support research on 4D scene understanding, multi-Lidar fusion, and velocity-aware perception and planning.

Problem

• 4D FMCW Lidar
• motion-aware perception
• autonomous driving
• velocity measurement
• dynamic scene understanding

Innovation

• 4D FMCW Lidar
• radial velocity
• motion-aware perception
• multi-modal dataset
• autonomous driving