Improving Local Feature Matching by Entropy-inspired Scale Adaptability and Flow-endowed Local Consistency

2026-04-08

AI Summary

This work addresses the sensitivity to scale discrepancies and insufficient local consistency in fine matching stages inherent in mutual nearest neighbor–based semi-dense image matching. To overcome these limitations, the authors propose a scale-adaptive matching method that extracts scale information from the score matrix and introduces an entropy-inspired, scale-aware matching module. Furthermore, fine matching is reformulated as a cascaded optical flow optimization problem, augmented with a gradient regularization loss that explicitly enforces local consistency. The proposed approach achieves substantial improvements in matching accuracy and robustness while maintaining extremely low computational overhead, demonstrating state-of-the-art performance across multiple downstream tasks.

Abstract

Recent semi-dense image matching methods have achieved remarkable success, but two long-standing issues still impair their performance. At the coarse stage, the over-exclusion issue of their mutual nearest neighbor (MNN) matching layer makes them struggle to handle cases with scale difference between images. To this end, we comprehensively revisit the matching mechanism and make a key observation that the hint concealed in the score matrix can be exploited to indicate the scale ratio. Based on this, we propose a scale-aware matching module which is exceptionally effective but introduces negligible overhead. At the fine stage, we point out that existing methods neglect the local consistency of final matches, which undermines their robustness. To this end, rather than independently predicting the correspondence for each source pixel, we reformulate the fine stage as a cascaded flow refinement problem and introduce a novel gradient loss to encourage local consistency of the flow field. Extensive experiments demonstrate that our novel matching pipeline, with these proposed modifications, achieves robust and accurate matching performance on downstream tasks.

Problem

• local feature matching
• scale difference
• local consistency
• mutual nearest neighbor
• image matching

Innovation

• scale adaptability
• local consistency
• feature matching
• flow refinement
• entropy-inspired