SGAD: Semantic and Geometric-aware Descriptor for Local Feature Matching

2025-08-04

AI Summary

Local feature matching suffers from low regional matching efficiency and heavy reliance on computationally expensive graph optimization. To address this, we propose the Semantic and Geometry-aware Descriptor Network (SGDN), an end-to-end trainable framework for direct region matching. SGDN introduces learnable regional descriptors, jointly supervised by classification and ranking losses, and incorporates a hierarchical containment-based redundancy filtering mechanism to enhance robustness. Geometric constraints are modeled via hierarchical containment graphs—bypassing explicit graph matching computation. Integrated with point-based matchers (e.g., LoFTR, ROMA), SGDN achieves both high accuracy and significant efficiency gains: 60× faster than MESA; outdoor pose estimation accuracy of 65.98 (surpassing DKM); and indoor AUC@5° improved by 7.39%, achieving state-of-the-art performance.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Computer Vision: Motion & Tracking
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Local feature matching remains a fundamental challenge in computer vision. Recent Area to Point Matching (A2PM) methods have improved matching accuracy. However, existing research based on this framework relies on inefficient pixel-level comparisons and complex graph matching that limit scalability. In this work, we introduce the Semantic and Geometric-aware Descriptor Network (SGAD), which fundamentally rethinks area-based matching by generating highly discriminative area descriptors that enable direct matching without complex graph optimization. This approach significantly improves both accuracy and efficiency of area matching. We further improve the performance of area matching through a novel supervision strategy that decomposes the area matching task into classification and ranking subtasks. Finally, we introduce the Hierarchical Containment Redundancy Filter (HCRF) to eliminate overlapping areas by analyzing containment graphs. SGAD demonstrates remarkable performance gains, reducing runtime by 60x (0.82s vs. 60.23s) compared to MESA. Extensive evaluations show consistent improvements across multiple point matchers: SGAD+LoFTR reduces runtime compared to DKM, while achieving higher accuracy (0.82s vs. 1.51s, 65.98 vs. 61.11) in outdoor pose estimation, and SGAD+ROMA delivers +7.39% AUC@5° in indoor pose estimation, establishing a new state-of-the-art.

Problem

• Improves local feature matching accuracy and efficiency
• Replaces complex graph optimization with direct area descriptors
• Eliminates overlapping areas using hierarchical containment analysis

Innovation

• Generates discriminative area descriptors for direct matching
• Uses classification and ranking subtasks for supervision
• Employs Hierarchical Containment Redundancy Filter for efficiency