COSTARR: Consolidated Open Set Technique with Attenuation for Robust Recognition

2025-08-01

AI Summary

In open-set recognition (OSR), existing methods rely on the “familiarity assumption” and thus struggle to robustly detect unknown classes. This paper proposes the “attenuation hypothesis”: during deep network training, small-weight features attenuated by gradients encode critical unfamiliarity cues, which—when synergistically combined with familiarity evidence—enhance known/unknown discrimination. To operationalize this, we design a unified scoring mechanism that jointly leverages raw deep features and Hadamard-product features derived from gradient-attenuated weights, integrated with a probabilistic open-set decision strategy. Evaluated on ImageNet-1K and multiple unknown datasets, our approach consistently outperforms state-of-the-art methods across diverse architectures (ViT, ConvNeXt, ResNet), demonstrating strong cross-architecture generalization. This work is the first to systematically uncover and exploit the discriminative value of attenuation-induced features, establishing both theoretical novelty and empirical superiority in OSR.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Handling novelty remains a key challenge in visual recognition systems. Existing open-set recognition (OSR) methods rely on the familiarity hypothesis, detecting novelty by the absence of familiar features. We propose a novel attenuation hypothesis: small weights learned during training attenuate features and serve a dual role-differentiating known classes while discarding information useful for distinguishing known from unknown classes. To leverage this overlooked information, we present COSTARR, a novel approach that combines both the requirement of familiar features and the lack of unfamiliar ones. We provide a probabilistic interpretation of the COSTARR score, linking it to the likelihood of correct classification and belonging in a known class. To determine the individual contributions of the pre- and post-attenuated features to COSTARR's performance, we conduct ablation studies that show both pre-attenuated deep features and the underutilized post-attenuated Hadamard product features are essential for improving OSR. Also, we evaluate COSTARR in a large-scale setting using ImageNet2012-1K as known data and NINCO, iNaturalist, OpenImage-O, and other datasets as unknowns, across multiple modern pre-trained architectures (ViTs, ConvNeXts, and ResNet). The experiments demonstrate that COSTARR generalizes effectively across various architectures and significantly outperforms prior state-of-the-art methods by incorporating previously discarded attenuation information, advancing open-set recognition capabilities.

Problem

• Handling novelty in visual recognition systems
• Improving open-set recognition using attenuation hypothesis
• Generalizing across architectures for robust recognition

Innovation

• Attenuation hypothesis for feature differentiation
• Combines familiar and unfamiliar feature analysis
• Utilizes pre- and post-attenuated features