🤖 AI Summary
This work addresses the vulnerability of large language models to prompt injection attacks under API and retrieval-augmented generation (RAG) architectures, which can lead to policy bypasses and unsafe outputs. The authors propose PCFI, a runtime defense mechanism that introduces, for the first time, role-origin- and priority-based prompt integrity control. It models prompts as structured segments—system, developer, user, and retrieved documents—and enforces priority-aware policies through a three-stage middleware pipeline. By integrating lexical heuristics, role-switching detection, and hierarchical policy enforcement, PCFI achieves a 100% attack interception rate with 0% false positives on a custom test suite, incurring only a median processing overhead of 0.04 milliseconds.
📝 Abstract
Large language models (LLMs) deployed behind APIs and retrieval-augmented generation (RAG) stacks are vulnerable to prompt injection attacks that may override system policies, subvert intended behavior, and induce unsafe outputs. Existing defenses often treat prompts as flat strings and rely on ad hoc filtering or static jailbreak detection. This paper proposes Prompt Control-Flow Integrity (PCFI), a priority-aware runtime defense that models each request as a structured composition of system, developer, user, and retrieved-document segments. PCFI applies a three-stage middleware pipeline, lexical heuristics, role-switch detection, and hierarchical policy enforcement, before forwarding requests to the backend LLM. We implement PCFI as a FastAPI-based gateway for deployed LLM APIs and evaluate it on a custom benchmark of synthetic and semi-realistic prompt-injection workloads. On the evaluated benchmark suite, PCFI intercepts all attack-labeled requests, maintains a 0% False Positive Rate, and introduces a median processing overhead of only 0.04 ms. These results suggest that provenance- and priority-aware prompt enforcement is a practical and lightweight defense for deployed LLM systems.