AdvPrefix: An Objective for Nuanced LLM Jailbreaks

2024-12-13

arXiv.org

AI Summary

Existing LLM jailbreaking attacks suffer from weak controllability, incomplete response generation, and rigid, non-adaptive optimization formats. To address these limitations, we propose AdvPrefix—a novel prefix-based objective function designed for fine-grained jailbreaking. AdvPrefix introduces the first model-adaptive prefix selection mechanism, which automatically identifies high-quality prefixes during the prefilling stage using a dual criterion: attack success rate and negative log-likelihood. It further enables multi-prefix collaborative optimization, departing from conventional fixed-prefix paradigms and exposing alignment models’ generalization vulnerabilities to unseen prefixes. AdvPrefix is fully compatible with mainstream optimization frameworks (e.g., GCG) and requires no model modification or retraining. Evaluated on Llama-3, AdvPrefix boosts GCG’s fine-grained jailbreaking success rate from 14% to 80%, demonstrating the critical impact of objective function design on jailbreaking efficacy.

Technology Category

• Search and Optimization: Adversarial Search
• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Many jailbreak attacks on large language models (LLMs) rely on a common objective: making the model respond with the prefix"Sure, here is (harmful request)". While straightforward, this objective has two limitations: limited control over model behaviors, often resulting in incomplete or unrealistic responses, and a rigid format that hinders optimization. To address these limitations, we introduce AdvPrefix, a new prefix-forcing objective that enables more nuanced control over model behavior while being easy to optimize. Our objective leverages model-dependent prefixes, automatically selected based on two criteria: high prefilling attack success rates and low negative log-likelihood. It can further simplify optimization by using multiple prefixes for a single user request. AdvPrefix can integrate seamlessly into existing jailbreak attacks to improve their performance for free. For example, simply replacing GCG attack's target prefixes with ours on Llama-3 improves nuanced attack success rates from 14% to 80%, suggesting that current alignment struggles to generalize to unseen prefixes. Our work demonstrates the importance of jailbreak objectives in achieving nuanced jailbreaks.

Problem

• Enhances control over LLM jailbreak response behaviors
• Improves optimization flexibility in prefix-based attack methods
• Addresses limitations of rigid prefix formats in jailbreak attacks

Innovation

• Selects model-dependent prefixes using success rates and log-likelihood
• Integrates plug-and-play into existing jailbreak attacks
• Improves nuanced attack success rates from 14% to 80%