ProtoConNet: Prototypical Augmentation and Alignment for Open-Set Few-Shot Image Classification

2025-07-15

AI Summary

To address insufficient generalization in open-set few-shot image classification caused by neglecting contextual information, this paper proposes the Prototype Enhancement and Alignment (PEA) framework. PEA constructs a contextual dictionary via clustering-driven data selection, integrates semantic context to refine object-centric features, and introduces a prototype alignment module to disentangle spurious context-object correlations while enhancing feature-space diversity. Furthermore, contrastive learning is incorporated to sharpen discriminative boundaries between known and unknown classes. Evaluated on two standard benchmarks, PEA achieves substantial improvements in both open-set recognition accuracy and unknown-class detection rate, consistently outperforming state-of-the-art methods. These results empirically validate the effectiveness of context-aware prototype learning for open-set generalization.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search
• Machine Learning: Semi-Supervised Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Open-set few-shot image classification aims to train models using a small amount of labeled data, enabling them to achieve good generalization when confronted with unknown environments. Existing methods mainly use visual information from a single image to learn class representations to distinguish known from unknown categories. However, these methods often overlook the benefits of integrating rich contextual information. To address this issue, this paper proposes a prototypical augmentation and alignment method, termed ProtoConNet, which incorporates background information from different samples to enhance the diversity of the feature space, breaking the spurious associations between context and image subjects in few-shot scenarios. Specifically, it consists of three main modules: the clustering-based data selection (CDS) module mines diverse data patterns while preserving core features; the contextual-enhanced semantic refinement (CSR) module builds a context dictionary to integrate into image representations, which boosts the model's robustness in various scenarios; and the prototypical alignment (PA) module reduces the gap between image representations and class prototypes, amplifying feature distances for known and unknown classes. Experimental results from two datasets verified that ProtoConNet enhances the effectiveness of representation learning in few-shot scenarios and identifies open-set samples, making it superior to existing methods.

Problem

• Enhance open-set few-shot image classification with limited labeled data
• Integrate contextual information to improve feature space diversity
• Reduce gap between image representations and class prototypes

Innovation

• Prototypical augmentation enhances feature space diversity
• Contextual integration boosts model robustness
• Prototypical alignment reduces representation-prototype gap