SecureLearn - An Attack-agnostic Defense for Multiclass Machine Learning Against Data Poisoning Attacks

2025-10-25

AI Summary

To address the lack of effective defenses for multi-class classifiers against data poisoning attacks, this paper proposes SecureLearn—a general-purpose defense framework. SecureLearn employs an attack-agnostic, two-tier mechanism: a lower tier performs training-set purification via data denoising, while an upper tier enhances model robustness through feature-oriented adversarial training—supporting both traditional classifiers (e.g., Random Forest, Naïve Bayes) and deep models (e.g., MLP). A novel three-dimensional evaluation matrix is introduced to systematically quantify defense efficacy. Extensive experiments on multiple public benchmarks demonstrate that, under 10%–20% poisoning rates, SecureLearn achieves classification accuracy exceeding 90% across all models, with recall and F1 scores above 75% (reaching 97% for neural networks)—significantly outperforming state-of-the-art methods. Crucially, SecureLearn breaks the dependency of existing defenses on specific attack types or model architectures, offering broad applicability and strong generalization.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Security and Privacy: Security and privacy of machine learning and AI applications

Abstract

Data poisoning attacks are a potential threat to machine learning (ML) models, aiming to manipulate training datasets to disrupt their performance. Existing defenses are mostly designed to mitigate specific poisoning attacks or are aligned with particular ML algorithms. Furthermore, most defenses are developed to secure deep neural networks or binary classifiers. However, traditional multiclass classifiers need attention to be secure from data poisoning attacks, as these models are significant in developing multi-modal applications. Therefore, this paper proposes SecureLearn, a two-layer attack-agnostic defense to defend multiclass models from poisoning attacks. It comprises two components of data sanitization and a new feature-oriented adversarial training. To ascertain the effectiveness of SecureLearn, we proposed a 3D evaluation matrix with three orthogonal dimensions: data poisoning attack, data sanitization and adversarial training. Benchmarking SecureLearn in a 3D matrix, a detailed analysis is conducted at different poisoning levels (10%-20%), particularly analysing accuracy, recall, F1-score, detection and correction rates, and false discovery rate. The experimentation is conducted for four ML algorithms, namely Random Forest (RF), Decision Tree (DT), Gaussian Naive Bayes (GNB) and Multilayer Perceptron (MLP), trained with three public datasets, against three poisoning attacks and compared with two existing mitigations. Our results highlight that SecureLearn is effective against the provided attacks. SecureLearn has strengthened resilience and adversarial robustness of traditional multiclass models and neural networks, confirming its generalization beyond algorithm-specific defenses. It consistently maintained accuracy above 90%, recall and F1-score above 75%. For neural networks, SecureLearn achieved 97% recall and F1-score against all selected poisoning attacks.

Problem

• Defending multiclass ML models against diverse data poisoning attacks
• Addressing limitations of algorithm-specific defenses for traditional classifiers
• Enhancing security for multi-modal applications using attack-agnostic protection

Innovation

• Two-layer attack-agnostic defense for multiclass models
• Combines data sanitization with feature-oriented adversarial training
• Uses 3D evaluation matrix to assess poisoning attack resilience