A Layered Security Framework Against Prompt Injection in RAG-Based Chatbots

2026-06-17

AI Summary

This work addresses the vulnerability of Retrieval-Augmented Generation (RAG) chatbots to both direct and indirect prompt injection attacks, noting that existing defenses are limited to isolated pipeline stages and fail to provide end-to-end protection. To bridge this gap, the authors propose the first holistic, three-tiered defense framework spanning the entire RAG pipeline: malicious queries are filtered at the input layer; the context assembly layer employs an instruction source prioritization mechanism to prevent system policies from being overwritten by adversarial documents; and the output layer integrates semantic and policy-based auditing. Implemented as a model-agnostic middleware, the architecture dynamically combines rule-based filters, fine-tuned classifiers, instruction provenance tracking, and semantic drift detection. Experiments demonstrate that the framework reduces attack success rates from 71.4% to 11.3% across GPT-4o, Llama 3, and Mistral 7B—outperforming the best single-layer baseline by 27.3 percentage points—with a false positive rate of 4.8% and a median latency overhead of 61.2 milliseconds.

Abstract

Prompt injection is ranked as the most critical vulnerability in large language model (LLM) deployments by the OWASP Top 10 for LLM Applications, yet existing defenses operate at isolated pipeline stages and remain incomplete. Input filters cannot inspect retrieved documents, while output monitors cannot prevent malicious payloads from reaching the model. Consequently, retrieval-augmented generation (RAG) chatbots remain vulnerable to indirect injection, where a poisoned knowledge-base document compromises every user whose query retrieves it. We present a three-layer framework that intercepts both direct and indirect prompt injection throughout the inference pipeline. Layer 1 screens user input using a rule-based pattern library and a fine-tuned semantic anomaly classifier. Layer 2 enforces a provenance-based instruction hierarchy during context assembly, preventing retrieved content from overriding operator policy. Layer 3 audits model output using a policy rule engine and semantic drift detector before delivery. A continuous audit loop aggregates structured logs and supports retraining to adapt the classifier to emerging attack patterns. The framework is model-agnostic and deploys as middleware without modifying the underlying LLM. Evaluation on 5,080 samples across GPT-4o, Llama 3, and Mistral 7B shows that the framework reduces Attack Success Rate (ASR) from 71.4\% to 11.3\%, outperforming the best single-layer baseline by 27.3 percentage points and a published guardrail system by 23.8 percentage points, while maintaining a 4.8\% false positive rate and a median latency overhead of 61.2 ms. Ablation studies confirm that all three layers provide complementary protection and that their combined effect exceeds the sum of individual contributions.

Problem

• prompt injection
• retrieval-augmented generation
• LLM security
• indirect injection
• chatbot vulnerability

Innovation

• prompt injection
• retrieval-augmented generation
• layered security framework
• semantic anomaly detection
• provenance-based instruction hierarchy