MambaGlue: Fast and Robust Local Feature Matching With Mamba

2025-02-01

AI Summary

Existing deep learning-based image matching methods struggle to balance speed and accuracy in local feature correspondence. To address this, we propose the first end-to-end local feature matching framework built upon the Mamba architecture. Our key contributions are: (1) the pioneering integration of the state-space model Mamba into local matching; (2) a novel MambaAttention mixer enabling joint local-global feature modeling; and (3) a lightweight MLP-based confidence regressor for dynamic selection of high-reliability correspondences. The framework jointly optimizes feature detection, description, and matching in a unified pipeline. Extensive experiments demonstrate state-of-the-art performance across multiple standard benchmarks: inference speed improves by 3.2× over Transformer-based baselines, matching accuracy increases by 12.7%, and the method achieves superior robustness and real-time capability.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Learning with Manifolds
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In recent years, robust matching methods using deep learning-based approaches have been actively studied and improved in computer vision tasks. However, there remains a persistent demand for both robust and fast matching techniques. To address this, we propose a novel Mamba-based local feature matching approach, called MambaGlue, where Mamba is an emerging state-of-the-art architecture rapidly gaining recognition for its superior speed in both training and inference, and promising performance compared with Transformer architectures. In particular, we propose two modules: a) MambaAttention mixer to simultaneously and selectively understand the local and global context through the Mamba-based self-attention structure and b) deep confidence score regressor, which is a multi-layer perceptron (MLP)-based architecture that evaluates a score indicating how confidently matching predictions correspond to the ground-truth correspondences. Consequently, our MambaGlue achieves a balance between robustness and efficiency in real-world applications. As verified on various public datasets, we demonstrate that our MambaGlue yields a substantial performance improvement over baseline approaches while maintaining fast inference speed. Our code will be available on https://github.com/url-kaist/MambaGlue

Problem

• Deep Learning Image Matching
• Speed-Accuracy Trade-off
• Local Feature Pairing

Innovation

• MambaAttention
• Depth Confidence Score Regressor
• MambaGlue