HeRCULES: Heterogeneous Radar Dataset in Complex Urban Environment for Multi-session Radar SLAM

2025-02-04

AI Summary

Existing radar SLAM datasets are predominantly limited to a single radar modality, hindering the exploitation of complementary advantages across heterogeneous radar sensors. To address this, we introduce the first heterogeneous multimodal radar SLAM dataset tailored for complex urban environments. It uniquely integrates 4D imaging radar, rotating LiDAR, FMCW LiDAR, IMU, GNSS, and multiple cameras—captured under diverse weather conditions, illumination levels, and dynamic traffic scenarios. The dataset supports multi-session and multi-robot localization and mapping, and provides millimeter-accurate ground-truth trajectories, repeated traversal paths, and fully calibrated cross-sensor extrinsics and intrinsics. A comprehensive open-source toolchain is included for FMCW signal processing, high-precision GNSS/INS calibration, and multimodal spatiotemporal synchronization. Over 100 km of road sequences are released, enabling substantial improvements in robustness and reproducibility of multimodal radar SLAM under challenging conditions such as rain and fog.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Recently, radars have been widely featured in robotics for their robustness in challenging weather conditions. Two commonly used radar types are spinning radars and phased-array radars, each offering distinct sensor characteristics. Existing datasets typically feature only a single type of radar, leading to the development of algorithms limited to that specific kind. In this work, we highlight that combining different radar types offers complementary advantages, which can be leveraged through a heterogeneous radar dataset. Moreover, this new dataset fosters research in multi-session and multi-robot scenarios where robots are equipped with different types of radars. In this context, we introduce the HeRCULES dataset, a comprehensive, multi-modal dataset with heterogeneous radars, FMCW LiDAR, IMU, GPS, and cameras. This is the first dataset to integrate 4D radar and spinning radar alongside FMCW LiDAR, offering unparalleled localization, mapping, and place recognition capabilities. The dataset covers diverse weather and lighting conditions and a range of urban traffic scenarios, enabling a comprehensive analysis across various environments. The sequence paths with multiple revisits and ground truth pose for each sensor enhance its suitability for place recognition research. We expect the HeRCULES dataset to facilitate odometry, mapping, place recognition, and sensor fusion research. The dataset and development tools are available at https://sites.google.com/view/herculesdataset.

Problem

• Heterogeneous radar dataset creation
• Multi-session radar SLAM research
• Sensor fusion in urban environments

Innovation

• Heterogeneous radar dataset
• Multi-session radar SLAM
• 4D and spinning radar integration