Cross3DReg: Towards a Large-scale Real-world Cross-source Point Cloud Registration Benchmark

2025-09-08

AI Summary

Cross-source point cloud registration faces two key challenges: scarcity of large-scale real-world data and substantial geometric/semantic discrepancies across multi-sensor point clouds, severely degrading feature matching robustness. To address these, we introduce the largest publicly available real-world multimodal point cloud registration dataset to date, comprising both rotating and hybrid semi-solid-state LiDAR scans. We further propose an end-to-end registration framework guided by overlap region prediction: leveraging unaligned images to identify salient regions, and designing a vision–geometry attention matching module that jointly fuses image texture and point cloud geometry to enhance cross-source feature consistency. Experiments demonstrate state-of-the-art performance on mainstream benchmarks—achieving 63.2% and 40.2% reductions in rotation and translation errors, respectively, and a 5.4% improvement in registration recall.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Security and Privacy: Data transparency and provenance

Abstract

Cross-source point cloud registration, which aims to align point cloud data from different sensors, is a fundamental task in 3D vision. However, compared to the same-source point cloud registration, cross-source registration faces two core challenges: the lack of publicly available large-scale real-world datasets for training the deep registration models, and the inherent differences in point clouds captured by multiple sensors. The diverse patterns induced by the sensors pose great challenges in robust and accurate point cloud feature extraction and matching, which negatively influence the registration accuracy. To advance research in this field, we construct Cross3DReg, the currently largest and real-world multi-modal cross-source point cloud registration dataset, which is collected by a rotating mechanical lidar and a hybrid semi-solid-state lidar, respectively. Moreover, we design an overlap-based cross-source registration framework, which utilizes unaligned images to predict the overlapping region between source and target point clouds, effectively filtering out redundant points in the irrelevant regions and significantly mitigating the interference caused by noise in non-overlapping areas. Then, a visual-geometric attention guided matching module is proposed to enhance the consistency of cross-source point cloud features by fusing image and geometric information to establish reliable correspondences and ultimately achieve accurate and robust registration. Extensive experiments show that our method achieves state-of-the-art registration performance. Our framework reduces the relative rotation error (RRE) and relative translation error (RTE) by $63.2%$ and $40.2%$, respectively, and improves the registration recall (RR) by $5.4%$, which validates its effectiveness in achieving accurate cross-source registration.

Problem

• Lack of large-scale real-world datasets for cross-source point cloud registration
• Inherent differences in point clouds from multiple sensors challenge feature extraction
• Noise in non-overlapping areas negatively impacts registration accuracy

Innovation

• Largest real-world multi-modal cross-source dataset
• Overlap-based registration framework filtering redundant points
• Visual-geometric attention guided matching module enhancing feature consistency