BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models

2024-08-23

AI Summary

This work addresses the backdoor attack threat to large language models (LLMs) in open-ended text generation—where adversaries manipulate models to produce malicious outputs via triggers. We introduce the first comprehensive benchmark for backdoor attacks and defenses tailored to LLM text generation, encompassing diverse attack modalities: data poisoning, weight tampering, hidden-state manipulation, and chain-of-thought hijacking. A standardized evaluation framework is proposed, integrating seven defense techniques. Through systematic analysis, we uncover failure mechanisms of backdoors and identify critical influencing factors. Experiments span eight attack strategies, seven realistic application scenarios, and six mainstream LLM architectures, yielding over 200 evaluation configurations. Our benchmark and methodology won First Prize in the SafetyBench competition hosted by the AI Safety Center, establishing the first rigorous foundation for studying backdoors in open-generation LLMs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

Generative large language models (LLMs) have achieved state-of-the-art results on a wide range of tasks, yet they remain susceptible to backdoor attacks: carefully crafted triggers in the input can manipulate the model to produce adversary-specified outputs. While prior research has predominantly focused on backdoor risks in vision and classification settings, the vulnerability of LLMs in open-ended text generation remains underexplored. To fill this gap, we introduce BackdoorLLM (Our BackdoorLLM benchmark was awarded First Prize in the SafetyBench competition, https://www.mlsafety.org/safebench/winners, organized by the Center for AI Safety, https://safe.ai/.), the first comprehensive benchmark for systematically evaluating backdoor threats in text-generation LLMs. BackdoorLLM provides: (i) a unified repository of benchmarks with a standardized training and evaluation pipeline; (ii) a diverse suite of attack modalities, including data poisoning, weight poisoning, hidden-state manipulation, and chain-of-thought hijacking; (iii) over 200 experiments spanning 8 distinct attack strategies, 7 real-world scenarios, and 6 model architectures; (iv) key insights into the factors that govern backdoor effectiveness and failure modes in LLMs; and (v) a defense toolkit encompassing 7 representative mitigation techniques. Our code and datasets are available at https://github.com/bboylyg/BackdoorLLM. We will continuously incorporate emerging attack and defense methodologies to support the research in advancing the safety and reliability of LLMs.

Problem

• Assessing backdoor vulnerabilities in text-generation LLMs
• Evaluating diverse attack strategies on LLM safety
• Developing defenses against backdoor attacks in LLMs

Innovation

• Comprehensive benchmark for LLM backdoor threats
• Diverse attack modalities and defense toolkit
• Standardized pipeline with extensive experimental evaluation