SHeRLoc: Synchronized Heterogeneous Radar Place Recognition for Cross-Modal Localization

2025-06-18

AI Summary

Heterogeneous radar systems (e.g., mmWave and FMCW radars) and cross-modal sensing (e.g., radar–LiDAR) suffer from significant domain shifts, poor generalization, and the absence of a unified representation framework for place recognition. Method: We propose the first deep learning-based place recognition method tailored for heterogeneous radar data. It aligns multi-source radar inputs via RCS-based polar coordinate matching; designs rotation-robust, multi-scale feature descriptors; introduces hierarchical optimal transport for feature aggregation and FOV-aware metric learning; and jointly leverages FFT-based similarity mining and adaptive-margin triplet loss. Contribution/Results: Evaluated on public benchmarks, our method improves recall@1 from <0.1 to 0.9—a full order-of-magnitude gain—and establishes, for the first time, a unified framework supporting both intra-radar heterogeneous and cross-modal radar–LiDAR place recognition.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Despite the growing adoption of radar in robotics, the majority of research has been confined to homogeneous sensor types, overlooking the integration and cross-modality challenges inherent in heterogeneous radar technologies. This leads to significant difficulties in generalizing across diverse radar data types, with modality-aware approaches that could leverage the complementary strengths of heterogeneous radar remaining unexplored. To bridge these gaps, we propose SHeRLoc, the first deep network tailored for heterogeneous radar, which utilizes RCS polar matching to align multimodal radar data. Our hierarchical optimal transport-based feature aggregation method generates rotationally robust multi-scale descriptors. By employing FFT-similarity-based data mining and adaptive margin-based triplet loss, SHeRLoc enables FOV-aware metric learning. SHeRLoc achieves an order of magnitude improvement in heterogeneous radar place recognition, increasing recall@1 from below 0.1 to 0.9 on a public dataset and outperforming state of-the-art methods. Also applicable to LiDAR, SHeRLoc paves the way for cross-modal place recognition and heterogeneous sensor SLAM. The source code will be available upon acceptance.

Problem

• Addressing cross-modality challenges in heterogeneous radar localization
• Improving generalization across diverse radar data types
• Enabling robust place recognition for heterogeneous sensors

Innovation

• Deep network for heterogeneous radar data
• Hierarchical optimal transport feature aggregation
• FFT-similarity data mining for metric learning