Granular Ball Twin Support Vector Machine

2024-10-07

IEEE Transactions on Neural Networks and Learning Systems

AI Summary

To address the three key limitations of Transductive Support Vector Machines (TSVMs)—computational inefficiency on large-scale data (due to matrix inversion), poor generalization (lacking Structural Risk Minimization, SRM), and weak noise robustness (sensitivity to label noise and outliers)—this paper proposes the Granular Ball Twin Support Vector Machine (GB-TSVM). GB-TSVM replaces raw samples with granular balls as fundamental modeling units, enabling a large-scale convex optimization formulation that avoids matrix inversion. It explicitly incorporates the SRM principle and an L₂-norm regularization term to jointly enhance robustness, generalization, and training efficiency. Extensive experiments on multi-class benchmark datasets from UCI, KEEL, and NDC demonstrate that GB-TSVM achieves several-fold speedup over TSVM and other baselines, significantly improves robustness to label noise, and attains statistically superior generalization performance. Overall, GB-TSVM offers enhanced efficiency, stability, and scalability for transductive learning.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Twin support vector machine (TSVM) is an emerging machine learning model with versatile applicability in classification and regression endeavors. Nevertheless, TSVM confronts noteworthy challenges: 1) the imperative demand for matrix inversions presents formidable obstacles to its efficiency and applicability on large-scale datasets; 2) the omission of the structural risk minimization (SRM) principle in its primal formulation heightens the vulnerability to overfitting risks; and 3) the TSVM exhibits a high susceptibility to noise and outliers and also demonstrates instability when subjected to resampling. In view of the aforementioned challenges, we propose the granular ball TSVM (GBTSVM). GBTSVM takes granular balls (GBs), rather than individual data points, as inputs to construct a classifier. These GBs, characterized by their coarser granularity, exhibit robustness to resampling and reduced susceptibility to the impact of noise and outliers. We further propose a novel large-scale GBTSVM (LS-GBTSVM). LS-GBTSVM's optimization formulation ensures two critical facets: 1) it eliminates the need for matrix inversions, streamlining the LS-GBTSVM's computational efficiency; and 2) it incorporates the SRM principle through the incorporation of regularization terms, effectively addressing the issue of overfitting. The proposed LS-GBTSVM exemplifies efficiency, scalability for large datasets, and robustness against noise and outliers. We conduct a comprehensive evaluation of the GBTSVM and LS-GBTSVM models on benchmark datasets from UCI and KEEL, both with and without the addition of label noise, and compared with existing baseline models. Furthermore, we extend our assessment to the large-scale NDC datasets to establish the practicality of the proposed models in such contexts. Our experimental findings and rigorous statistical analyses affirm the superior generalization prowess of the proposed GBTSVM and LS-GBTSVM models compared to the baseline models. The source code of the proposed GBTSVM and LS-GBTSVM models are available at https://github.com/mtanveer1/GBTSVM.

Problem

• Improving TSVM efficiency by avoiding matrix inversions
• Enhancing TSVM robustness against noise and outliers
• Incorporating SRM principle to prevent overfitting in TSVM

Innovation

• Uses granular balls for robust classification
• Eliminates matrix inversions for efficiency
• Incorporates SRM principle to prevent overfitting