One Class Restricted Kernel Machines

2025-02-11

AI Summary

Restricted Kernel Machines (RKMs) suffer from poor robustness and limited generalization capability in the presence of outliers and data contamination. Method: This paper proposes One-Class Restricted Kernel Machines (OCRKM), the first approach to integrate one-class classification principles into the RKM framework. OCRKM constructs a non-probabilistic latent-variable model based on an RBM-like energy function, enabling joint modeling of visible and hidden layers while avoiding strong assumptions about data distribution. It performs unsupervised anomaly detection without labeled data, enhancing robustness against noise and contamination. Results: Extensive experiments on multiple UCI benchmark datasets demonstrate that OCRKM consistently outperforms baseline methods—including RKMs and Least Squares Support Vector Machines—in anomaly detection accuracy. Statistical significance tests confirm its substantially superior generalization performance.

Technology Category

• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Kernel Methods
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Restricted kernel machines (RKMs) have demonstrated a significant impact in enhancing generalization ability in the field of machine learning. Recent studies have introduced various methods within the RKM framework, combining kernel functions with the least squares support vector machine (LSSVM) in a manner similar to the energy function of restricted boltzmann machines (RBM), such that a better performance can be achieved. However, RKM's efficacy can be compromised by the presence of outliers and other forms of contamination within the dataset. These anomalies can skew the learning process, leading to less accurate and reliable outcomes. To address this critical issue and to ensure the robustness of the model, we propose the novel one-class RKM (OCRKM). In the framework of OCRKM, we employ an energy function akin to that of the RBM, which integrates both visible and hidden variables in a nonprobabilistic setting. The formulation of the proposed OCRKM facilitates the seamless integration of one-class classification method with the RKM, enhancing its capability to detect outliers and anomalies effectively. The proposed OCRKM model is evaluated over UCI benchmark datasets. Experimental findings and statistical analyses consistently emphasize the superior generalization capabilities of the proposed OCRKM model over baseline models across all scenarios.

Problem

• Enhance outlier detection in RKMs
• Improve RKM robustness against data anomalies
• Integrate one-class classification with RKM framework

Innovation

• One-Class RKM for outlier detection
• RBM-inspired energy function integration
• Enhanced generalization with UCI datasets