Bridging Robustness and Generalization Against Word Substitution Attacks in NLP via the Growth Bound Matrix Approach

2025-07-14

AI Summary

To address the insufficient robustness of NLP models against synonym substitution–based adversarial attacks, this paper presents the first systematic evaluation of the vulnerability of state-space models (SSMs), specifically the S4 architecture. We propose a novel regularization method—Growth-Bounded Matrix (GBM) regularization—that uniformly enhances both adversarial robustness and clean-data generalization across diverse sequential architectures, including LSTM, S4, and CNN, by explicitly constraining the model’s output sensitivity to input perturbations. Unlike prior work predominantly focused on feedforward or CNN-based models, our study bridges a critical gap by investigating robustness in recurrent and state-space models, achieving joint optimization of robustness and accuracy. Extensive experiments on multiple benchmark datasets demonstrate that GBM regularization improves adversarial robustness by up to 8.8%, significantly outperforming state-of-the-art defense methods.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Despite advancements in Natural Language Processing (NLP), models remain vulnerable to adversarial attacks, such as synonym substitutions. While prior work has focused on improving robustness for feed-forward and convolutional architectures, the robustness of recurrent networks and modern state space models (SSMs), such as S4, remains understudied. These architectures pose unique challenges due to their sequential processing and complex parameter dynamics. In this paper, we introduce a novel regularization technique based on Growth Bound Matrices (GBM) to improve NLP model robustness by reducing the impact of input perturbations on model outputs. We focus on computing the GBM for three architectures: Long Short-Term Memory (LSTM), State Space models (S4), and Convolutional Neural Networks (CNN). Our method aims to (1) enhance resilience against word substitution attacks, (2) improve generalization on clean text, and (3) providing the first systematic analysis of SSM (S4) robustness. Extensive experiments across multiple architectures and benchmark datasets demonstrate that our method improves adversarial robustness by up to 8.8% over existing baselines. These results highlight the effectiveness of our approach, outperforming several state-of-the-art methods in adversarial defense. Codes are available at https://github.com/BouriMohammed/GBM

Problem

• Improving NLP model robustness against word substitution attacks
• Enhancing generalization on clean text for sequential models
• Analyzing robustness of State Space Models (S4) systematically

Innovation

• Introduces Growth Bound Matrix regularization technique
• Enhances robustness against word substitution attacks
• First systematic analysis of S4 model robustness