OpenSlot: Mixed Open-set Recognition with Object-centric Learning

2024-07-02

arXiv.org

AI Summary

This work addresses the realistic challenge in Hybrid Open-Set Recognition (H-OSR) where test images contain both known and unknown classes under severe *hyper-label shift*—a scenario wherein class priors and label semantics deviate significantly between training and testing. We formally model hyper-label shift in H-OSR for the first time. To tackle it, we propose an object-centric slot attention architecture incorporating a noise-robust slot (NRS) mechanism to mitigate semantic misalignment, enabling multi-semantic disentangled representation learning and joint classification-localization training—without requiring bounding-box annotations for open-set object localization. Our method achieves state-of-the-art performance on both hybrid and standard OSR benchmarks. It significantly outperforms existing approaches in hyper-label shift detection and open-set object detection, demonstrating strong generalization and improved suitability for deployment in real-world open environments.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Object Detection & Categorization
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Existing open-set recognition (OSR) studies typically assume that each image contains only one class label, with the unknown test set (negative) having a disjoint label space from the known test set (positive), a scenario referred to as full-label shift. This paper introduces the mixed OSR problem, where test images contain multiple class semantics, with both known and unknown classes co-occurring in the negatives, leading to a more complex super-label shift that better reflects real-world scenarios. To tackle this challenge, we propose the OpenSlot framework, based on object-centric learning, which uses slot features to represent diverse class semantics and generate class predictions. The proposed anti-noise slot (ANS) technique helps mitigate the impact of noise (invalid or background) slots during classification training, addressing the semantic misalignment between class predictions and ground truth. We evaluate OpenSlot on both mixed and conventional OSR benchmarks. Without elaborate designs, our method not only excels existing approaches in detecting super-label shifts across OSR tasks, but also achieves state-of-the-art performance on conventional benchmarks. Meanwhile, OpenSlot can localize class objects without using bounding boxes during training, demonstrating competitive performance in open-set object detection and potential for generalization.

Problem

• Mixed Open Set Recognition
• Multi-Class Identification
• Hyper-Label Shift

Innovation

• OpenSlot
• Hybrid Open-Set Recognition
• Slot Representation Learning