Semantic Correspondence: Unified Benchmarking and a Strong Baseline

2025-05-23

AI Summary

Semantic correspondence aims to match semantically consistent keypoints across images, yet has long suffered from the absence of systematic surveys and unified evaluation protocols. This paper introduces the first comprehensive, reproducible semantic correspondence benchmark. Our contributions are threefold: (1) We propose the first hierarchical taxonomy of methodologies, clarifying the technical evolution and conceptual relationships among existing approaches; (2) We design a lightweight, efficient baseline model that achieves state-of-the-art performance on major benchmarks including PF-PASCAL and SPair-71k; (3) We establish cross-dataset standardized evaluation protocols and perform component-wise ablation studies, substantially improving experimental comparability and reproducibility. All code, configuration files, and evaluation tools are publicly released under an open-source license, enabling plug-and-play reproduction.

Technology Category

• Computer Vision: Image and Video Retrieval
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

Establishing semantic correspondence is a challenging task in computer vision, aiming to match keypoints with the same semantic information across different images. Benefiting from the rapid development of deep learning, remarkable progress has been made over the past decade. However, a comprehensive review and analysis of this task remains absent. In this paper, we present the first extensive survey of semantic correspondence methods. We first propose a taxonomy to classify existing methods based on the type of their method designs. These methods are then categorized accordingly, and we provide a detailed analysis of each approach. Furthermore, we aggregate and summarize the results of methods in literature across various benchmarks into a unified comparative table, with detailed configurations to highlight performance variations. Additionally, to provide a detailed understanding on existing methods for semantic matching, we thoroughly conduct controlled experiments to analyse the effectiveness of the components of different methods. Finally, we propose a simple yet effective baseline that achieves state-of-the-art performance on multiple benchmarks, providing a solid foundation for future research in this field. We hope this survey serves as a comprehensive reference and consolidated baseline for future development. Code is publicly available at: https://github.com/Visual-AI/Semantic-Correspondence.

Problem

• Establish semantic correspondence across different images
• Review and analyze existing semantic correspondence methods
• Propose a strong baseline for future research

Innovation

• Proposes first extensive semantic correspondence survey
• Conducts controlled experiments on method components
• Introduces simple effective baseline for benchmarks