An Engorgio Prompt Makes Large Language Model Babble on

2024-12-27

AI Summary

This work exposes a critical vulnerability of large language models (LLMs) during inference to computational-cost attacks—such as Engorgio—where adversarial prompts artificially inflate token generation length, exacerbating latency and resource consumption, thereby compromising service availability in resource-constrained deployments. To address this, the authors propose the first white-box prompt optimization framework grounded in predictive trajectory distribution modeling. It introduces a novel, stable suppression loss for autoregressive interruption tokens, jointly leveraging parameterized trajectory modeling and gradient-guided optimization to efficiently synthesize highly disruptive prompts. Extensive evaluation across 13 open-source LLMs (ranging from 125M to 30B parameters) demonstrates that the generated prompts increase output length by 2–13×, consistently approaching model-specific maximum generation limits. These results empirically substantiate a severe, practical availability threat to real-world LLM services.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Prompt Engineering / Prompting
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Auto-regressive large language models (LLMs) have yielded impressive performance in many real-world tasks. However, the new paradigm of these LLMs also exposes novel threats. In this paper, we explore their vulnerability to inference cost attacks, where a malicious user crafts Engorgio prompts to intentionally increase the computation cost and latency of the inference process. We design Engorgio, a novel methodology, to efficiently generate adversarial Engorgio prompts to affect the target LLM's service availability. Engorgio has the following two technical contributions. (1) We employ a parameterized distribution to track LLMs' prediction trajectory. (2) Targeting the auto-regressive nature of LLMs' inference process, we propose novel loss functions to stably suppress the appearance of thetoken, whose occurrence will interrupt the LLM's generation process. We conduct extensive experiments on 13 open-sourced LLMs with parameters ranging from 125M to 30B. The results show that Engorgio prompts can successfully induce LLMs to generate abnormally long outputs (i.e., roughly 2-13$ imes$ longer to reach 90%+ of the output length limit) in a white-box scenario and our real-world experiment demonstrates Engergio's threat to LLM service with limited computing resources. The code is accessible at https://github.com/jianshuod/Engorgio-prompt.

Problem

• Large Language Models
• Stability under Attack
• Computational Load

Innovation

• Engorgio
• Resource Consumption Attack
• Large Language Model Stability