Ex-Sim(3)-Reg: 2D-3D Correspondence Pruning via Extended Sim(3) Registration

2026-08-28

AI Summary

本文针对图像到点云配准中的离群点问题，提出了一种通过扩展Sim(3)配准来修剪2D-3D对应关系的新方法Ex-Sim(3)-Reg，有效提升了配准召回率。

Abstract

Learning-based image-to-point-cloud (I2P) registration has garnered increasing attention in recent years. Nevertheless, existing methods still struggle with severe outliers under challenging scenarios with unseen, low-inlier, or distorted cases. A fast and robust 2D-3D correspondence pruning method is therefore highly desirable. Recently, a promising scheme lifts 2D-3D correspondences to 3D-3D correspondences using depth priors, casting correspondence pruning as a Sim(3) registration problem. However, depth priors estimated from monocular images are inherently noisy, which undermines the reliability of this scheme. In this paper, to explicitly model non-negligible depth noise, we reformulate correspondence pruning as an extended Sim(3) registration problem and propose a simple yet effective pruning algorithm termed Ex-Sim(3)-Reg. We further provide a theoretical analysis to justify the effectiveness of our method. Extensive experiments on the 7-Scenes, RGBD-V2, ScanNet, and TUM datasets demonstrate that Ex-Sim(3)-Reg achieves up to \textbf{24.7\% improvement} in registration recall over state-of-the-art baseline methods. Code is released at github.com/anpei96/ex-sim3-demo

Problem

• 2D-3D Correspondence
• Outliers
• Depth Priors
• Sim(3) Registration

Innovation

• Extended Sim(3) Registration
• 2D-3D Correspondence Pruning
• Depth Noise