Introducing 'Inside' Out of Distribution

2024-07-05

arXiv.org

AI Summary

Existing out-of-distribution (OOD) research predominantly focuses on extrapolative (“outside”) anomalies while overlooking interpolative (“inside”) in-distribution anomalies—i.e., samples that reside within the support of the training distribution yet are semantically anomalous. Method: This work formally defines and empirically validates the “inside OOD” concept, proposing a two-dimensional analytical framework to distinguish inside from outside OOD. Leveraging statistical distribution analysis, geometric modeling in feature space, and multi-model robustness evaluation, we systematically characterize their co-occurrence patterns and differential impacts on model behavior. Results: We demonstrate that inside OOD triggers latent, progressive performance degradation, whereas outside OOD induces sharp confidence collapse. Our framework bridges a critical gap in conventional OOD detection, providing both theoretical grounding and empirical evidence for designing targeted defense mechanisms against distinct OOD categories.

Technology Category

• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Detecting and understanding out-of-distribution (OOD) samples is crucial in machine learning (ML) to ensure reliable model performance. Current OOD studies, in general, and in the context of ML, in particular, primarily focus on extrapolatory OOD (outside), neglecting potential cases of interpolatory OOD (inside). This study introduces a novel perspective on OOD by suggesting OOD can be divided into inside and outside cases. In addition, following this framework, we examine the inside-outside OOD profiles of datasets and their impact on ML model performance. Our analysis shows that different inside-outside OOD profiles lead to nuanced declines in ML model performance, highlighting the importance of distinguishing between these two cases for developing effective counter-OOD methods.

Problem

• Distinguishing inside versus outside out-of-distribution detection cases
• Examining unique impacts of inside-outside OOD profiles on ML performance
• Addressing neglect of interpolatory inside OOD in current studies

Innovation

• Introducing inside-outside OOD classification
• Using normalized RMSE and F1 metrics
• Analyzing unique performance degradation effects