Enhanced Feature Based Granular Ball Twin Support Vector Machine

2024-10-08

arXiv.org

AI Summary

To address the poor generalization, weak robustness, and limited scalability of SVMs under noisy and large-scale data conditions, this paper proposes the GB-RVFL-TSVM framework. It employs granular balls (GBs) as coarse-grained input units, constructs an enhanced feature space via random projection and nonlinear activation within a Random Vector Functional Link (RVFL) architecture, and deploys Twin Support Vector Machines (TSVM) to learn two nonparallel hyperplanes in this space. This work is the first to integrate granular ball representation with the RVFL structure into TSVM, simultaneously improving robustness, generalization, and scalability. Extensive experiments across 27 benchmark datasets from UCI, KEEL, and NDC demonstrate that GB-RVFL-TSVM significantly outperforms SVM, TSVM, and RVFL-SVM baselines. Statistically significant improvements are observed in generalization performance under label noise and resampling perturbations, as well as in scalability on large-scale datasets.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Intelligent Robots: Learning & Optimization for ROB
• Computer Vision: Representation Learning for Vision

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

In this paper, we propose enhanced feature based granular ball twin support vector machine (EF-GBTSVM). EF-GBTSVM employs the coarse granularity of granular balls (GBs) as input rather than individual data samples. The GBs are mapped to the feature space of the hidden layer using random projection followed by the utilization of a non-linear activation function. The concatenation of original and hidden features derived from the centers of GBs gives rise to an enhanced feature space, commonly referred to as the random vector functional link (RVFL) space. This space encapsulates nuanced feature information to GBs. Further, we employ twin support vector machine (TSVM) in the RVFL space for classification. TSVM generates the two non-parallel hyperplanes in the enhanced feature space, which improves the generalization performance of the proposed EF-GBTSVM model. Moreover, the coarser granularity of the GBs enables the proposed EF-GBTSVM model to exhibit robustness to resampling, showcasing reduced susceptibility to the impact of noise and outliers. We undertake a thorough evaluation of the proposed EF-GBTSVM model on benchmark UCI and KEEL datasets. This evaluation encompasses scenarios with and without the inclusion of label noise. Moreover, experiments using NDC datasets further emphasize the proposed model's ability to handle large datasets. Experimental results, supported by thorough statistical analyses, demonstrate that the proposed EF-GBTSVM model significantly outperforms the baseline models in terms of generalization capabilities, scalability, and robustness. The source code for the proposed EF-GBTSVM model, along with additional results and further details, can be accessed at https://github.com/mtanveer1/EF-GBTSVM.

Problem

• Improves classification using enhanced feature space
• Enhances robustness to noise and outliers
• Handles large datasets efficiently

Innovation

• Granular balls as input
• Random vector functional link space
• Twin support vector machine classification