S-GBT: Smooth Growth Bound Tensor for Certified Robustness Against Word Substitution Attacks in NLP
This work addresses the limited certified robustness of existing natural language processing models against word substitution attacks, which typically account only for first-order sensitivity while neglecting second-order curvature information. To overcome this limitation, the authors propose Smoothed Growth Bound Tensors (S-GBT), a novel framework that incorporates quadratic terms of output variation into certified robustness analysis for the first time. By imposing element-wise constraints on the Hessian matrix, S-GBT constructs a second-order robustness bound and introduces a joint regularization term that simultaneously optimizes both first- and second-order sensitivities during training. Implemented on LSTM and CNN architectures, the method integrates Hessian-bound estimation and second-order Taylor expansion directly into the training objective. Experiments demonstrate that S-GBT achieves up to a 23.4% improvement in certified robust accuracy across multiple benchmark datasets while maintaining strong clean accuracy.