Generalized Class Discovery in Instance Segmentation

2025-02-12

AI Summary

This work addresses Generalized Category Discovery (GCD) for instance segmentation under long-tailed class distributions, tackling the dual challenges of discovering novel categories in unlabeled data and unifying segmentation for both known and unknown classes. We propose three key innovations: (1) an Instance-level Temperature Allocation (ITA) mechanism to enhance contrastive learning discriminability for tail instances; (2) a dynamic class-reliability–guided pseudo-labeling criterion to mitigate pseudo-label sparsity for tail classes; and (3) a lightweight soft-attention module to improve cross-class feature disentanglement. Our method is seamlessly integrated into the Mask R-CNN framework without requiring additional annotations or architectural modifications. Evaluated on two long-tailed benchmarks—COCO$_{half}$+LVIS and LVIS+Visual Genome—our approach significantly outperforms state-of-the-art methods, achieving substantial gains in tail-class discovery rate and segmentation AP. To our knowledge, this is the first work to systematically resolve discriminative imbalance and pseudo-label unreliability in GCD under long-tailed settings.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

This work addresses the task of generalized class discovery (GCD) in instance segmentation. The goal is to discover novel classes and obtain a model capable of segmenting instances of both known and novel categories, given labeled and unlabeled data. Since the real world contains numerous objects with long-tailed distributions, the instance distribution for each class is inherently imbalanced. To address the imbalanced distributions, we propose an instance-wise temperature assignment (ITA) method for contrastive learning and class-wise reliability criteria for pseudo-labels. The ITA method relaxes instance discrimination for samples belonging to head classes to enhance GCD. The reliability criteria are to avoid excluding most pseudo-labels for tail classes when training an instance segmentation network using pseudo-labels from GCD. Additionally, we propose dynamically adjusting the criteria to leverage diverse samples in the early stages while relying only on reliable pseudo-labels in the later stages. We also introduce an efficient soft attention module to encode object-specific representations for GCD. Finally, we evaluate our proposed method by conducting experiments on two settings: COCO$_{half}$ + LVIS and LVIS + Visual Genome. The experimental results demonstrate that the proposed method outperforms previous state-of-the-art methods.

Problem

• Discover novel classes in segmentation
• Address imbalanced instance distributions
• Enhance generalized class discovery

Innovation

• Instance-wise temperature assignment method
• Class-wise reliability criteria
• Efficient soft attention module