Rethinking Backdoor Detection Evaluation for Language Models

2024-08-31

arXiv.org

AI Summary

Existing backdoor detectors for language models—e.g., Spectral Signatures and Activation Clustering—exhibit critically low robustness in realistic settings, as their performance heavily depends on the poisoning intensity (e.g., number of poisoned epochs, learning rate, poison data ratio) used during backdoor injection; recall drops by over 40% under non-default, “overly strong” or “overly weak” poisoning configurations. Method: We conduct controlled backdoor injection experiments to systematically evaluate the generalization of mainstream detectors across a multidimensional poisoning-intensity space. Contribution/Results: This work provides the first empirical evidence that current evaluation benchmarks—due to their reliance on narrow, default training strategies—lack realism and challenge. It exposes detector sensitivity to poisoning intensity as a fundamental flaw and advocates for a new evaluation paradigm featuring greater diversity, robustness, and real-world relevance.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Backdoor attacks, in which a model behaves maliciously when given an attacker-specified trigger, pose a major security risk for practitioners who depend on publicly released language models. Backdoor detection methods aim to detect whether a released model contains a backdoor, so that practitioners can avoid such vulnerabilities. While existing backdoor detection methods have high accuracy in detecting backdoored models on standard benchmarks, it is unclear whether they can robustly identify backdoors in the wild. In this paper, we examine the robustness of backdoor detectors by manipulating different factors during backdoor planting. We find that the success of existing methods highly depends on how intensely the model is trained on poisoned data during backdoor planting. Specifically, backdoors planted with either more aggressive or more conservative training are significantly more difficult to detect than the default ones. Our results highlight a lack of robustness of existing backdoor detectors and the limitations in current benchmark construction.

Problem

• Evaluating robustness of language model backdoor detection methods
• Assessing detection effectiveness against varied backdoor training intensities
• Identifying limitations in current backdoor detection benchmark construction

Innovation

• Evaluates backdoor detection robustness
• Manipulates training intensity factors
• Identifies limitations in current benchmarks