LongPIBench: A Long-Context Benchmark for Prompt Injection

2026-08-28

AI Summary

本文通过引入LongPIBench，解决了长上下文环境中提示注入攻击的防御评估问题。该基准覆盖四个实际应用场景，并展示了现有防御方法的不足。

Abstract

Prompt injection attacks pose a serious security risk to large language models in real-world applications. However, existing prompt injection benchmarks primarily focus on short-context inputs, leaving the attacks and defenses in long-context settings largely unexplored. This gap leads to a substantial overestimation of the effectiveness of current defenses. In this paper, we bridge the gap by introducing LongPIBench, a long-context benchmark for prompt injection covering 4 realistic application scenarios: paper peer review, resume screening, code review, and email summary. For each scenario, we construct a synthetic dataset and a real-world dataset, with context lengths ranging from thousands to tens of thousands of tokens. The evaluation results on LongPIBench reveal significant vulnerabilities of prompt injection defenses under long-context settings: even simple heuristic prompt injection attacks achieve high success rates and frequently bypass state-of-the-art defenses. We hope LongPIBench can serve as a practical benchmark for systematically evaluating prompt injection defenses in realistic long-context scenarios.

Problem

• Prompt Injection
• Long-Context
• Security Risk
• Benchmark
• Large Language Models

Innovation

• Long-Context Benchmark
• Prompt Injection
• Real-World Scenarios
• Defense Vulnerabilities