🤖 AI Summary
This work reveals a critical vulnerability of Transformer models for tabular data to backdoor attacks during training. To address this, we propose the first in-bounds backdoor attack paradigm tailored to tabular data: it constructs imperceptible, in-distribution triggers via feature-space-constrained optimization—ensuring triggers remain within natural data bounds and induce no visible perturbations—thereby achieving high stealth and attack success rates. We are the first to demonstrate that Transformers exhibit extreme sensitivity to minute feature-level perturbations, and that such attacks transfer effectively to non-Transformer models including XGBoost and DeepFM. Across multiple benchmark tabular datasets, our attack achieves up to 100% attack success rate while degrading clean-sample accuracy by less than 0.5%. Furthermore, we develop a model-agnostic defense evaluation framework based on Spectral Signatures, empirically establishing it as the most effective current method for backdoor detection.
📝 Abstract
Deep Neural Networks (DNNs) have shown great promise in various domains. Alongside these developments, vulnerabilities associated with DNN training, such as backdoor attacks, are a significant concern. These attacks involve the subtle insertion of triggers during model training, allowing for manipulated predictions. More recently, DNNs for tabular data have gained increasing attention due to the rise of transformer models. Our research presents a comprehensive analysis of backdoor attacks on tabular data using DNNs, mainly focusing on transformers. We also propose a novel approach for trigger construction: an in-bounds attack, which provides excellent attack performance while maintaining stealthiness. Through systematic experimentation across benchmark datasets, we uncover that transformer-based DNNs for tabular data are highly susceptible to backdoor attacks, even with minimal feature value alterations. We also verify that our attack can be generalized to other models, like XGBoost and DeepFM. Our results demonstrate up to 100% attack success rate with negligible clean accuracy drop. Furthermore, we evaluate several defenses against these attacks, identifying Spectral Signatures as the most effective. Nevertheless, our findings highlight the need to develop tabular data-specific countermeasures to defend against backdoor attacks.
