Good Keypoints for the Two-View Geometry Estimation Problem

2025-03-24

AI Summary

This work addresses the critical impact of local feature quality on downstream geometric estimation performance in two-view geometry. For the first time, it theoretically models keypoint quality, identifying repeatability and low measurement error as the two fundamental determinants of geometric estimation accuracy. To this end, we propose BoNeSS-ST—a novel keypoint detector featuring: (i) a sub-pixel refinement mechanism for high-precision localization; (ii) a self-supervised keypoint scoring function; and (iii) enhanced robustness to low-salience regions. BoNeSS-ST is jointly trained on both homography and fundamental matrix estimation tasks to promote geometric consistency. Evaluated on planar homography and epipolar geometry estimation benchmarks, BoNeSS-ST significantly outperforms existing self-supervised detectors, achieving state-of-the-art performance in both matching quality and geometric estimation accuracy.

Technology Category

• Intelligent Robots: State Estimation
• Computer Vision: Other Foundations of Computer Vision
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

Local features are essential to many modern downstream applications. Therefore, it is of interest to determine the properties of local features that contribute to the downstream performance for a better design of feature detectors and descriptors. In our work, we propose a new theoretical model for scoring feature points (keypoints) in the context of the two-view geometry estimation problem. The model determines two properties that a good keypoint for solving the homography estimation problem should have: be repeatable and have a small expected measurement error. This result provides key insights into why maximizing the number of correspondences doesn't always lead to better homography estimation accuracy. We use the developed model to design a method that detects keypoints that benefit the homography estimation introducing the Bounded NeSS-ST (BoNeSS-ST) keypoint detector. The novelty of BoNeSS-ST comes from strong theoretical foundations, a more accurate keypoint scoring due to subpixel refinement and a cost designed for superior robustness to low saliency keypoints. As a result, BoNeSS-ST outperforms prior self-supervised local feature detectors in both planar homography and epipolar geometry estimation problems.

Problem

• Identify keypoint properties for better two-view geometry estimation
• Develop a model for scoring keypoints in homography estimation
• Design a keypoint detector improving homography and epipolar geometry accuracy

Innovation

• Theoretical model for scoring keypoints
• Subpixel refinement for accurate scoring
• Cost design for robustness to low saliency