Enhancing Robustness and Efficiency of Least Square Twin SVM via Granular Computing

2024-10-22

arXiv.org

AI Summary

To address the sensitivity to noise, poor generalization, low computational efficiency, and violation of the Structural Risk Minimization (SRM) principle in Least Squares Twin Support Vector Machines (LSTSVM), this paper introduces granular computing into the LSTSVM framework for the first time, proposing a robust and efficient Granular Ball-based LSTSVM (GBLSTSVM) and its large-scale extension, LS-GBLSTSVM. The proposed methods replace raw samples with granular balls for modeling, explicitly incorporate an SRM-motivated regularization term, and avoid matrix inversion—thereby enhancing robustness, numerical stability, and scalability. Extensive experiments on 32 benchmark datasets from UCI, KEEL, and NDC demonstrate that GBLSTSVM and LS-GBLSTSVM consistently outperform LSTSVM and other baselines: training speed is accelerated by up to 5.3×, and noise robustness is significantly improved.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Intelligent Robots: Learning & Optimization for ROB
• Natural Language Processing: Safety and Robustness

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

In the domain of machine learning, least square twin support vector machine (LSTSVM) stands out as one of the state-of-the-art models. However, LSTSVM suffers from sensitivity to noise and outliers, overlooking the SRM principle and instability in resampling. Moreover, its computational complexity and reliance on matrix inversions hinder the efficient processing of large datasets. As a remedy to the aforementioned challenges, we propose the robust granular ball LSTSVM (GBLSTSVM). GBLSTSVM is trained using granular balls instead of original data points. The core of a granular ball is found at its center, where it encapsulates all the pertinent information of the data points within the ball of specified radius. To improve scalability and efficiency, we further introduce the large-scale GBLSTSVM (LS-GBLSTSVM), which incorporates the SRM principle through regularization terms. Experiments are performed on UCI, KEEL, and NDC benchmark datasets; both the proposed GBLSTSVM and LS-GBLSTSVM models consistently outperform the baseline models.

Problem

• Improve robustness of LSTSVM
• Enhance efficiency in large datasets
• Address sensitivity to noise and outliers

Innovation

• Granular ball replaces original data points
• Incorporates SRM principle via regularization
• Improves scalability and efficiency significantly