SceneGlue: Scene-Aware Transformer for Feature Matching without Scene-Level Annotation

2026-04-15

AI Summary

This work addresses the limitations of traditional local feature matching methods, which struggle to achieve high accuracy due to their inability to model non-local cross-view scene context. To overcome this, we propose SceneGlue, a novel framework that, for the first time, integrates implicit parallel attention with an explicit visibility-aware Transformer to enable scene-aware feature matching without requiring scene-level annotations. By exchanging local descriptors both within and across images and explicitly modeling visible regions across views, SceneGlue substantially enhances the interpretability and robustness of matches. Extensive experiments demonstrate that our method consistently outperforms state-of-the-art approaches across multiple tasks—including homography estimation, pose estimation, image matching, and visual localization—validating its effectiveness and strong generalization capability.

Technology Category

• Computer Vision: Scene Analysis & Understanding
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Data transparency and provenance

Abstract

Local feature matching plays a critical role in understanding the correspondence between cross-view images. However, traditional methods are constrained by the inherent local nature of feature descriptors, limiting their ability to capture non-local scene information that is essential for accurate cross-view correspondence. In this paper, we introduce SceneGlue, a scene-aware feature matching framework designed to overcome these limitations. SceneGlue leverages a hybridizable matching paradigm that integrates implicit parallel attention and explicit cross-view visibility estimation. The parallel attention mechanism simultaneously exchanges information among local descriptors within and across images, enhancing the scene's global context. To further enrich the scene awareness, we propose the Visibility Transformer, which explicitly categorizes features into visible and invisible regions, providing an understanding of cross-view scene visibility. By combining explicit and implicit scene-level awareness, SceneGlue effectively compensates for the local descriptor constraints. Notably, SceneGlue is trained using only local feature matches, without requiring scene-level groundtruth annotations. This scene-aware approach not only improves accuracy and robustness but also enhances interpretability compared to traditional methods. Extensive experiments on applications such as homography estimation, pose estimation, image matching, and visual localization validate SceneGlue's superior performance. The source code is available at https://github.com/songlin-du/SceneGlue.

Problem

• feature matching
• cross-view correspondence
• scene awareness
• local descriptors
• non-local information

Innovation

• scene-aware matching
• parallel attention
• Visibility Transformer
• cross-view correspondence
• annotation-free training