A Unified Approach Towards Active Learning and Out-of-Distribution Detection

2024-05-18

Trans. Mach. Learn. Res.

AI Summary

Open-world deep learning requires joint handling of active learning (AL) and out-of-distribution (OOD) detection, yet existing approaches treat them independently, leading to poor cross-task generalization. To address this, we propose SISOM—the first unified framework that jointly models AL and OOD detection via a shared feature-space distance metric, without introducing auxiliary networks or loss functions. SISOM enables end-to-end optimization and synergistic representation learning for both tasks. Evaluated on the OpenOOD benchmark, it achieves two state-of-the-art (SOTA) results and one second-best performance across its three major subsets. On three standard AL benchmarks, SISOM consistently ranks first, demonstrating superior query efficiency and robustness in open-world settings. The framework significantly improves annotation efficiency while maintaining strong OOD discrimination, advancing practical deployment of adaptive deep learning systems under distributional shift.

Technology Category

• Machine Learning: Active Learning
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

When applying deep learning models in open-world scenarios, active learning (AL) strategies are crucial for identifying label candidates from a nearly infinite amount of unlabeled data. In this context, robust out-of-distribution (OOD) detection mechanisms are essential for handling data outside the target distribution of the application. However, current works investigate both problems separately. In this work, we introduce SISOM as the first unified solution for both AL and OOD detection. By leveraging feature space distance metrics SISOM combines the strengths of the currently independent tasks to solve both effectively. We conduct extensive experiments showing the problems arising when migrating between both tasks. In these evaluations SISOM underlined its effectiveness by achieving first place in two of the widely used OpenOOD benchmarks and second place in the remaining one. In AL, SISOM outperforms others and delivers top-1 performance in three benchmarks

Problem

• Unified approach for active learning and OOD detection
• Solving both problems jointly using feature space metrics
• Improving performance on OpenOOD benchmarks and AL tasks

Innovation

• Unified solution combining active learning and OOD detection
• Leverages feature space distance metrics for both tasks
• Achieves top performance in OpenOOD and AL benchmarks