GeoUniPR: A Geometry-Consistent Unified Framework for Cross-Modal Place Recognition

2026-08-09

AI Summary

This work addresses the challenge of large modality gaps and alignment difficulties in cross-modal (vision–LiDAR) place recognition by proposing a unified representation learning approach that avoids complex alignment or multi-stage training. The method projects LiDAR point clouds into the camera view to construct a multi-channel geometrically consistent image encoding depth, intensity, and surface normals, which is then processed by a shared ViT encoder to enable matching within a unified embedding space. By integrating parameter-efficient fine-tuning with a novel Spatial Consistency InfoNCE loss (SC-InfoNCE), the proposed framework achieves state-of-the-art performance for both intra- and cross-modal place recognition on KITTI and KITTI-360, while demonstrating strong generalization across datasets.

Abstract

Cross-modal place recognition (CMPR) aims to identify the same location across heterogeneous sensing modalities, such as vision and LiDAR. Existing methods commonly bridge the modality gap using complex alignment modules, multi-stage training, or full fine-tuning of pretrained backbones. In this work, we revisit CMPR from the perspective of geometric consistency and propose GeoUniPR, a unified and concise geometry-consistent framework. GeoUniPR reduces cross-modal discrepancy at the representation level by projecting LiDAR point clouds into the camera perspective to construct Geometry-Consistent depth image views (DIV), which establish direct RGB-LiDAR correspondence. We further augment DIV with native LiDAR cues, including intensity and surface-normal information, yielding a multi-channel geometric representation that improves structural consistency. Based on this representation, GeoUniPR learns a unified embedding space using two modality-specific ViT-based encoders with identical architectures, trained through parameter-efficient adaptation without auxiliary alignment modules, multi-stage training, or full backbone fine-tuning. In addition, we introduce Spatially-Consistent InfoNCE (SC-InfoNCE), a CMPR-specific contrastive objective that suppresses distance-induced false negatives under spatial continuity. Extensive experiments on KITTI and KITTI-360 demonstrate that GeoUniPR achieves state-of-the-art (SOTA) performance in both same-modal and cross-modal place recognition, with strong cross-dataset generalization.

Problem

• cross-modal place recognition
• modality gap
• geometric consistency
• LiDAR
• vision

Innovation

• Geometry-Consistent
• Cross-Modal Place Recognition
• Depth Image Views
• Parameter-Efficient Adaptation
• Spatially-Consistent InfoNCE