Kernel-Free Universum Quadratic Surface Twin Support Vector Machines for Imbalanced Data

2024-12-02

arXiv.org

AI Summary

To address the challenges of modeling minority classes in imbalanced binary classification and the susceptibility of conventional linear/kernel-based methods to class bias, this paper proposes the kernel-free Quadratic Twin Support Vector Machine with Universum (QTSVM-U)—the first framework integrating Universum learning with a quadratic surface decision mechanism. QTSVM-U abandons kernel tricks and instead employs flexible quadratic surfaces to capture complex decision boundaries. It incorporates Universum samples to impose discriminative constraints that enhance minority-class separation, and adopts an imbalance-aware reweighting strategy to further improve minority-class recognition. Experiments on four synthetic and multiple benchmark imbalanced datasets demonstrate that QTSVM-U achieves average improvements of 8.3% in minority-class recall and 6.7% in G-mean over state-of-the-art methods. The approach combines strong representational capacity, inherent interpretability, and computational efficiency.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Binary classification tasks with imbalanced classes pose significant challenges in machine learning. Traditional classifiers often struggle to accurately capture the characteristics of the minority class, resulting in biased models with subpar predictive performance. In this paper, we introduce a novel approach to tackle this issue by leveraging Universum points to support the minority class within quadratic twin support vector machine models. Unlike traditional classifiers, our models utilize quadratic surfaces instead of hyperplanes for binary classification, providing greater flexibility in modeling complex decision boundaries. By incorporating Universum points, our approach enhances classification accuracy and generalization performance on imbalanced datasets. We generated four artificial datasets to demonstrate the flexibility of the proposed methods. Additionally, we validated the effectiveness of our approach through empirical evaluations on benchmark datasets, showing superior performance compared to conventional classifiers and existing methods for imbalanced classification.

Problem

• Addressing imbalanced binary classification using Universum points
• Employing quadratic surfaces instead of hyperplanes for decision boundaries
• Enhancing accuracy and generalization on imbalanced datasets

Innovation

• Uses quadratic surfaces for binary classification
• Incorporates Universum points to support minority class
• Enhances accuracy and generalization on imbalanced data