An Iterative Task-Driven Framework for Resilient LiDAR Place Recognition in Adverse Weather

2025-04-21

AI Summary

Under adverse weather conditions (e.g., rain, snow, fog), LiDAR point clouds suffer severe degradation, leading to drastic performance drops in place recognition (LPR). To address this, we propose ITDNet—an iterative task-driven network enabling end-to-end joint optimization of point cloud restoration and LPR for the first time. Our key contributions are: (1) an alternating optimization mechanism between the LiDAR Degradation Restoration (LDR) and LPR modules; (2) a Dual-Domain Mixer (DDM) and a Semantic-Aware Generator (SAG) ensuring structurally consistent and robust reconstruction; and (3) a Multi-Frequency Transformer (MFT) coupled with Wavelet Pyramid NetVLAD (WPN) to enhance frequency-spatial fusion and multi-scale discriminative descriptor learning. ITDNet achieves state-of-the-art performance on Weather-KITTI, Boreas, and our newly introduced Weather-Apollo dataset, significantly improving recall rates and localization robustness across all weather conditions. Code and datasets will be publicly released.

Technology Category

• Computer Vision: Language and Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

LiDAR place recognition (LPR) plays a vital role in autonomous navigation. However, existing LPR methods struggle to maintain robustness under adverse weather conditions such as rain, snow, and fog, where weather-induced noise and point cloud degradation impair LiDAR reliability and perception accuracy. To tackle these challenges, we propose an Iterative Task-Driven Framework (ITDNet), which integrates a LiDAR Data Restoration (LDR) module and a LiDAR Place Recognition (LPR) module through an iterative learning strategy. These modules are jointly trained end-to-end, with alternating optimization to enhance performance. The core rationale of ITDNet is to leverage the LDR module to recover the corrupted point clouds while preserving structural consistency with clean data, thereby improving LPR accuracy in adverse weather. Simultaneously, the LPR task provides feature pseudo-labels to guide the LDR module's training, aligning it more effectively with the LPR task. To achieve this, we first design a task-driven LPR loss and a reconstruction loss to jointly supervise the optimization of the LDR module. Furthermore, for the LDR module, we propose a Dual-Domain Mixer (DDM) block for frequency-spatial feature fusion and a Semantic-Aware Generator (SAG) block for semantic-guided restoration. In addition, for the LPR module, we introduce a Multi-Frequency Transformer (MFT) block and a Wavelet Pyramid NetVLAD (WPN) block to aggregate multi-scale, robust global descriptors. Finally, extensive experiments on the Weather-KITTI, Boreas, and our proposed Weather-Apollo datasets demonstrate that, demonstrate that ITDNet outperforms existing LPR methods, achieving state-of-the-art performance in adverse weather. The datasets and code will be made publicly available at https://github.com/Grandzxw/ITDNet.

Problem

• Enhance LiDAR place recognition robustness in adverse weather
• Restore corrupted point clouds while preserving structural consistency
• Improve perception accuracy under rain, snow, and fog conditions

Innovation

• Iterative learning integrates LDR and LPR modules
• Dual-Domain Mixer fuses frequency-spatial features
• Multi-Frequency Transformer aggregates robust descriptors