Kernel PCA for Out-of-Distribution Detection: Non-Linear Kernel Selections and Approximations

2025-05-21

AI Summary

To address the insufficient reliability of out-of-distribution (OoD) detection in deep neural networks, this paper proposes a nonlinear feature subspace modeling method based on kernel principal component analysis (KPCA), leveraging reconstruction errors of in-distribution (InD) samples to discriminate OoD inputs. Our key contributions are: (1) a novel Cosine-Gaussian composite kernel function, explicitly capturing two nonlinear patterns strongly correlated with InD/OoD discrepancies; and (2) a confidence-weighted randomized Fourier features approximation algorithm, balancing accuracy and computational efficiency. Evaluated on multiple benchmark datasets, the proposed method achieves state-of-the-art performance in reconstruction-error-based OoD detection metrics, with significantly lower computational overhead than existing approaches. It substantially improves both detection accuracy and robustness against diverse OoD inputs, including adversarial and natural distribution shifts.

Technology Category

• Machine Learning: Kernel Methods
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Computer Vision: Other Foundations of Computer Vision

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Out-of-Distribution (OoD) detection is vital for the reliability of deep neural networks, the key of which lies in effectively characterizing the disparities between OoD and In-Distribution (InD) data. In this work, such disparities are exploited through a fresh perspective of non-linear feature subspace. That is, a discriminative non-linear subspace is learned from InD features to capture representative patterns of InD, while informative patterns of OoD features cannot be well captured in such a subspace due to their different distribution. Grounded on this perspective, we exploit the deviations of InD and OoD features in such a non-linear subspace for effective OoD detection. To be specific, we leverage the framework of Kernel Principal Component Analysis (KPCA) to attain the discriminative non-linear subspace and deploy the reconstruction error on such subspace to distinguish InD and OoD data. Two challenges emerge: (i) the learning of an effective non-linear subspace, i.e., the selection of kernel function in KPCA, and (ii) the computation of the kernel matrix with large-scale InD data. For the former, we reveal two vital non-linear patterns that closely relate to the InD-OoD disparity, leading to the establishment of a Cosine-Gaussian kernel for constructing the subspace. For the latter, we introduce two techniques to approximate the Cosine-Gaussian kernel with significantly cheap computations. In particular, our approximation is further tailored by incorporating the InD data confidence, which is demonstrated to promote the learning of discriminative subspaces for OoD data. Our study presents new insights into the non-linear feature subspace for OoD detection and contributes practical explorations on the associated kernel design and efficient computations, yielding a KPCA detection method with distinctively improved efficacy and efficiency.

Problem

• Detecting Out-of-Distribution data using non-linear Kernel PCA
• Selecting optimal kernel functions for effective subspace learning
• Approximating kernel matrices for efficient large-scale computations

Innovation

• Uses Kernel PCA for non-linear subspace learning
• Introduces Cosine-Gaussian kernel for OoD detection
• Approximates kernel matrix for efficient computation