Attacking Large Language Models with Projected Gradient Descent

2024-02-14

arXiv.org

AI Summary

Existing LLM alignment methods are vulnerable to adversarial prompt attacks; however, mainstream discrete optimization-based attacks require over 100,000 model queries, imposing prohibitive computational overhead and hindering quantitative evaluation and adversarial training. To address this, we propose the first continuous adversarial attack framework for LLMs based on projected gradient descent (PGD) in the input embedding space. Our method introduces a continuous relaxation of discrete prompts, enforced by gradient clipping and projection constraints that rigorously bound relaxation error. Empirically, it achieves comparable attack success rates while reducing model queries to approximately 10,000—yielding a tenfold speedup over state-of-the-art discrete methods. This substantially improves attack efficiency and reproducibility. Our work establishes a practical, scalable paradigm for LLM robustness analysis and defense research.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Adversarial Search
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Current LLM alignment methods are readily broken through specifically crafted adversarial prompts. While crafting adversarial prompts using discrete optimization is highly effective, such attacks typically use more than 100,000 LLM calls. This high computational cost makes them unsuitable for, e.g., quantitative analyses and adversarial training. To remedy this, we revisit Projected Gradient Descent (PGD) on the continuously relaxed input prompt. Although previous attempts with ordinary gradient-based attacks largely failed, we show that carefully controlling the error introduced by the continuous relaxation tremendously boosts their efficacy. Our PGD for LLMs is up to one order of magnitude faster than state-of-the-art discrete optimization to achieve the same devastating attack results.

Problem

• Adversarial prompts break current LLM alignment methods.
• Discrete optimization attacks require excessive computational resources.
• Projected Gradient Descent improves attack efficiency on LLMs.

Innovation

• Uses Projected Gradient Descent for attacks
• Controls error in continuous relaxation effectively
• Significantly faster than discrete optimization methods