Similarity-Aware Selective State-Space Modeling for Semantic Correspondence

2025-09-29

AI Summary

Semantic image correspondence requires modeling long-range inter-image dependencies, yet conventional approaches struggle to capture complex structural relationships. State-of-the-art methods based on 4D correlation volumes suffer from prohibitive computational cost, limiting scalability to high-resolution inputs and large receptive fields. To address this, we propose MambaMatcher—the first method to introduce selective state space models (SSMs) into semantic correspondence. We design a similarity-aware selective scanning mechanism that efficiently models the 4D correlation volume in linear complexity, enabling global dependency learning over high-resolution features. Our approach breaks the accuracy-efficiency trade-off: it achieves state-of-the-art performance on standard benchmarks including SPair-71k and PF-PASCAL, significantly outperforming both feature-matching and correlation-based methods. This demonstrates the strong representational capacity and practical viability of SSMs for visual correspondence tasks.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Establishing semantic correspondences between images is a fundamental yet challenging task in computer vision. Traditional feature-metric methods enhance visual features but may miss complex inter-correlation relationships, while recent correlation-metric approaches are hindered by high computational costs due to processing 4D correlation maps. We introduce MambaMatcher, a novel method that overcomes these limitations by efficiently modeling high-dimensional correlations using selective state-space models (SSMs). By implementing a similarity-aware selective scan mechanism adapted from Mamba's linear-complexity algorithm, MambaMatcher refines the 4D correlation map effectively without compromising feature map resolution or receptive field. Experiments on standard semantic correspondence benchmarks demonstrate that MambaMatcher achieves state-of-the-art performance.

Problem

• Establishing semantic correspondences between images
• Overcoming limitations of traditional feature-metric methods
• Reducing computational costs of correlation-metric approaches

Innovation

• Selective state-space models for high-dimensional correlation modeling
• Similarity-aware selective scan mechanism for efficient processing
• Linear-complexity algorithm maintaining resolution and receptive field