🤖 AI Summary
Image set classification (ISC) faces two key challenges: insufficient feature representation learning and non-adaptive inter-set similarity measurement—particularly limiting performance in few-shot settings. To address these, we propose a novel few-shot ISC framework that synergistically integrates deep learning with classical collaborative representation. Our approach is the first to embed class-specific collaborative representation (CSCR) into an end-to-end deep network, enabling adaptive concept-level feature refinement and joint optimization of inter-set distances. The framework comprises three core components: a fully convolutional feature extractor, a global feature aggregator, and a CSCR-based metric learning module, augmented by a newly designed CSCR contrastive loss. Extensive experiments on multiple mainstream few-shot ISC benchmarks demonstrate significant improvements over state-of-the-art methods, validating both effectiveness and robustness.
📝 Abstract
Image set classification (ISC), which can be viewed as a task of comparing similarities between sets consisting of unordered heterogeneous images with variable quantities and qualities, has attracted growing research attention in recent years. How to learn effective feature representations and how to explore the similarities between different image sets are two key yet challenging issues in this field. However, existing traditional ISC methods classify image sets based on raw pixel features, ignoring the importance of feature learning. Existing deep ISC methods can learn deep features, but they fail to adaptively adjust the features when measuring set distances, resulting in limited performance in few-shot ISC. To address the above issues, this paper combines traditional ISC methods with deep models and proposes a novel few-shot ISC approach called Deep Class-specific Collaborative Representation (DCSCR) network to simultaneously learn the frame- and concept-level feature representations of each image set and the distance similarities between different sets. Specifically, DCSCR consists of a fully convolutional deep feature extractor module, a global feature learning module, and a class-specific collaborative representation-based metric learning module. The deep feature extractor and global feature learning modules are used to learn (local and global) frame-level feature representations, while the class-specific collaborative representation-based metric learning module is exploit to adaptively learn the concept-level feature representation of each image set and thus obtain the distance similarities between different sets by developing a new CSCR-based contrastive loss function. Extensive experiments on several well-known few-shot ISC datasets demonstrate the effectiveness of the proposed method compared with some state-of-the-art image set classification algorithms.