Searching from Area to Point: A Hierarchical Framework for Semantic-Geometric Combined Feature Matching

2023-04-29

arXiv.org

AI Summary

To address the accuracy degradation in large-scale image feature matching caused by coarse search spaces, this paper proposes a hierarchical “Area-to-Point Matching” (A2PM) framework. A2PM first localizes semantically salient candidate matching regions and then performs fine-grained point-level matching within those regions. Its core innovation lies in introducing semantic region matching as the initial search space for point matching—marking the first such integration—and designing a Semantic-Geometric Alignment Module (SGAM) that jointly leverages semantic priors and geometric consistency constraints to enhance region matching robustness. The framework incorporates Transformer-based feature extraction, geometric verification, and optimization modules. Evaluated on large-scale point matching and pose estimation benchmarks, A2PM significantly outperforms existing Transformer-based matchers, effectively overcoming their inherent accuracy bottlenecks.

Abstract

Feature matching is a crucial technique in computer vision. A unified perspective for this task is to treat it as a searching problem, aiming at an efficient search strategy to narrow the search space to point matches between images. One of the key aspects of search strategy is the search space, which in current approaches is not carefully defined, resulting in limited matching accuracy. This paper, thus, pays attention to the search space and proposes to set the initial search space for point matching as the matched image areas containing prominent semantic, named semantic area matches. This search space favors point matching by salient features and alleviates the accuracy limitation in recent Transformer-based matching methods. To achieve this search space, we introduce a hierarchical feature matching framework: Area to Point Matching (A2PM), to first find semantic area matches between images and later perform point matching on area matches. We further propose Semantic and Geometry Area Matching (SGAM) method to realize this framework, which utilizes semantic prior and geometry consistency to establish accurate area matches between images. By integrating SGAM with off-the-shelf state-of-the-art matchers, our method, adopting the A2PM framework, achieves encouraging precision improvements in massive point matching and pose estimation experiments.

Problem

• Feature Matching
• Computer Vision
• Accuracy Improvement

Innovation

• Hierarchical Feature Matching
• Pattern Semantics and Shape Integration
• Pose Estimation Enhancement