Security--Fidelity Tradeoffs: The Hidden Cost of Prompt Injection Defense

2026-06-29

AI Summary

This study addresses the inherent trade-off between security and fidelity in large language models when defending against indirect prompt injection attacks. Such defenses often suppress untrusted input, inadvertently degrading performance on tasks requiring high output faithfulness, such as translation and text editing. The work presents SecFid, the first evaluation benchmark capable of distinguishing whether a model executes, retains, or ignores injected content. Through large-scale experiments encompassing 1,168 samples across 48 configurations, decision-theoretic analysis, and multi-model comparisons, the authors quantitatively demonstrate the tension between these objectives: the highest-fidelity model achieves 96.5% fidelity but only 47.8% security, whereas the most secure model attains 99.3% security at the cost of fidelity dropping to 71.0%–73.9%, confirming that simultaneously optimizing both dimensions remains fundamentally challenging.

Abstract

We identify a security-fidelity tradeoff in defending LLMs against indirect prompt injection: defenses resist injected instructions largely by suppressing untrusted text, which corrupts tasks that must preserve it, such as translation and document editing. Attack-success metrics cannot see this, because a model that ignores an injection and one that faithfully processes it as data score identically. We introduce SecFid, a benchmark built so that executing an injection, processing it as data, and ignoring it produce distinguishable outputs. This makes fidelity measurable and exposes a frontier: across 1,168 examples and 48 configurations, no model or defense achieves both objectives. The highest-fidelity model reaches 96.5% fidelity at 47.8% security, while the most secure defenses invert this, at 99.3% security but only 71.0%-73.9% fidelity. Even defenses with identical security differ in how they earn it: some repair hijacks into faithful processing, others simply suppress benign content. A decision-theoretic analysis shows why no fixed choice can be right everywhere: the correct behavior is not a property of the defense but of the deployment, set by its relative cost of a hijack versus a dropped span. Security alone therefore measures only half of robustness, and reporting it without fidelity hides the price at which it was bought.

Problem

• prompt injection
• security-fidelity tradeoff
• large language models
• defense robustness
• output fidelity

Innovation

• security-fidelity tradeoff
• prompt injection defense
• SecFid benchmark
• large language models
• robustness evaluation