Perturbations in the Orthogonal Complement Subspace for Efficient Out-of-Distribution Detection

2025-11-02

AI Summary

To address out-of-distribution (OOD) detection for deep learning models in open-world settings, this paper proposes P-OCS: a perturbation-based one-class scoring method that applies a single-step, constrained perturbation exclusively within the orthogonal complement subspace of the in-distribution (ID) feature principal component analysis (PCA) subspace—requiring neither model retraining nor external data. Its core innovation lies in rigorously confining the perturbation to this PCA-derived orthogonal complement and integrating an energy-based scoring function with gradient projection to yield highly discriminative OOD scores. Theoretically grounded and computationally efficient, P-OCS incurs minimal overhead—only one forward and one backward pass. Extensive evaluation across diverse architectures and benchmark datasets demonstrates state-of-the-art performance, significantly surpassing existing training-free OOD detection methods while preserving the original network architecture.

Technology Category

• Machine Learning: Feature Construction/Reformulation
• Computer Vision: Other Foundations of Computer Vision
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Out-of-distribution (OOD) detection is essential for deploying deep learning models in open-world environments. Existing approaches, such as energy-based scoring and gradient-projection methods, typically rely on high-dimensional representations to separate in-distribution (ID) and OOD samples. We introduce P-OCS (Perturbations in the Orthogonal Complement Subspace), a lightweight and theoretically grounded method that operates in the orthogonal complement of the principal subspace defined by ID features. P-OCS applies a single projected perturbation restricted to this complementary subspace, enhancing subtle ID-OOD distinctions while preserving the geometry of ID representations. We show that a one-step update is sufficient in the small-perturbation regime and provide convergence guarantees for the resulting detection score. Experiments across multiple architectures and datasets demonstrate that P-OCS achieves state-of-the-art OOD detection with negligible computational cost and without requiring model retraining, access to OOD data, or changes to model architecture.

Problem

• Detecting out-of-distribution samples for reliable deep learning deployment
• Enhancing ID-OOD distinction via orthogonal complement subspace perturbations
• Achieving efficient OOD detection without retraining or architectural changes

Innovation

• Perturbations in orthogonal complement subspace enhance detection
• One-step update with convergence guarantees for efficiency
• Lightweight method requiring no retraining or OOD data