PartCo: Part-Level Correspondence Priors Enhance Category Discovery

2025-09-26

AI Summary

Existing generalized category discovery (GCD) methods overly rely on global image representations and semantic labels, neglecting local fine-grained cues and thus struggling to discriminate semantically similar unseen categories. To address this, we propose PartCo, a part-level correspondence prior framework that— for the first time—models geometric and semantic correspondences among local parts as structured priors to explicitly uncover subtle inter-class distinctions. PartCo introduces a learnable part-matching mechanism that jointly optimizes local feature alignment and global semantic consistency, seamlessly integrating into standard GCD pipelines. Evaluated on CUB, AWA2, and SUN benchmarks, PartCo consistently outperforms state-of-the-art methods, achieving average improvements of 3.2–5.8 percentage points in unseen-class discovery accuracy. By leveraging interpretable part-level structure, PartCo establishes a new GCD paradigm that enhances both discriminability and explainability.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Generalized Category Discovery (GCD) aims to identify both known and novel categories within unlabeled data by leveraging a set of labeled examples from known categories. Existing GCD methods primarily depend on semantic labels and global image representations, often overlooking the detailed part-level cues that are crucial for distinguishing closely related categories. In this paper, we introduce PartCo, short for Part-Level Correspondence Prior, a novel framework that enhances category discovery by incorporating part-level visual feature correspondences. By leveraging part-level relationships, PartCo captures finer-grained semantic structures, enabling a more nuanced understanding of category relationships. Importantly, PartCo seamlessly integrates with existing GCD methods without requiring significant modifications. Our extensive experiments on multiple benchmark datasets demonstrate that PartCo significantly improves the performance of current GCD approaches, achieving state-of-the-art results by bridging the gap between semantic labels and part-level visual compositions, thereby setting new benchmarks for GCD. Project page: https://visual-ai.github.io/partco

Problem

• Identifies known and novel categories in unlabeled data
• Overcomes reliance on global image representations and semantic labels
• Incorporates part-level visual correspondences to distinguish related categories

Innovation

• PartCo uses part-level correspondence priors
• Integrates part-level visual feature correspondences
• Enhances category discovery with finer-grained structures