Vulnerability of Natural Language Classifiers to Evolutionary Generated Adversarial Text

2026-06-25

AI Summary

Natural language classifiers are vulnerable to semantics-preserving adversarial attacks in black-box settings, yet existing approaches suffer from limited efficiency and effectiveness. This work proposes GAversary, a novel method that, for the first time, integrates GloVe word embeddings into the mutation operator of a genetic algorithm to generate highly deceptive adversarial examples that maintain semantic similarity—requiring access only to the model’s logit outputs. Evaluated across multiple benchmark datasets, GAversary drastically reduces the target model’s accuracy from 76.8% to 5.8%, significantly outperforming state-of-the-art black-box attack methods such as BAE and A2T in terms of attack success rate.

Abstract

Deep learning models have achieved impressive performance across various fields but remain vulnerable to adversarial inputs, particularly in NLP, where such attacks can have significant real-world consequences. Adversarial attacks often involve small, semantically similar token replacements to fool NLP models, and recent methods have become more precise by targeting specific vulnerable words, often by exploiting some level of access to the model's internal structure. This paper proposes GAversary, a hybrid Genetic Algorithm (GA) to generate adversarial attacks on natural language models. The GA is able to treat the target model as a black box, requiring only the logit value output by the model to guide the search. GAversary differs from GAs previously proposed for this problem by using GloVe embeddings to propose word replacements (the mutation operator) to improve the semantic similarity of the adversarial examples. GAversary is applied to several benchmark data sets and well-known target models. GAversary is able to substantially reduce the target model's accuracy on test data compared to the BAE and A2T attacks compared against (in the best case, reducing a 76.8% accuracy to 5.8%, compared to BAE's 27.6%). The trade-off is that GAversary perturbs just under twice as many words as the other two methods, with a slightly lower semantic similarity to the original text and around a 5% increase in run-time.

Problem

• adversarial text
• natural language classifiers
• black-box attacks
• semantic similarity
• vulnerability

Innovation

• Genetic Algorithm
• Adversarial Text
• Black-box Attack
• GloVe Embeddings
• Natural Language Classifiers