Traceback of Poisoning Attacks to Retrieval-Augmented Generation

2025-04-22

The Web Conference

AI Summary

Retrieval-augmented generation (RAG) systems are vulnerable to knowledge base poisoning attacks, where adversaries inject malicious documents to manipulate large language models (LLMs) into generating incorrect or harmful outputs; existing inference-time defenses exhibit limited robustness against sophisticated, multi-step attacks. Method: We propose RAGForensics, the first forensics-inspired, traceable defense framework for RAG security. It integrates subset-based retrieval, LLM-driven iterative prompt-based detection, and a feedback refinement mechanism to precisely attribute poisoned content to its source document(s). Crucially, it requires no model architecture modification or retraining. Contribution/Results: RAGForensics achieves strong generalizability across diverse RAG configurations and attack types. Evaluated on multiple benchmarks, it significantly outperforms state-of-the-art defenses in溯源 accuracy—enabling, for the first time, fully automated, high-precision localization of poisoning sources within RAG pipelines.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large language models (LLMs) integrated with retrieval-augmented generation (RAG) systems improve accuracy by leveraging external knowledge sources. However, recent research has revealed RAG's susceptibility to poisoning attacks, where the attacker injects poisoned texts into the knowledge database, leading to attacker-desired responses. Existing defenses, which predominantly focus on inference-time mitigation, have proven insufficient against sophisticated attacks. In this paper, we introduce RAGForensics, the first traceback system for RAG, designed to identify poisoned texts within the knowledge database that are responsible for the attacks. RAGForensics operates iteratively, first retrieving a subset of texts from the database and then utilizing a specially crafted prompt to guide an LLM in detecting potential poisoning texts. Empirical evaluations across multiple datasets demonstrate the effectiveness of RAGForensics against state-of-the-art poisoning attacks. This work pioneers the traceback of poisoned texts in RAG systems, providing a practical and promising defense mechanism to enhance their security.

Problem

• Identify poisoned texts in RAG knowledge databases
• Detect poisoning attacks causing attacker-desired responses
• Enhance RAG security against sophisticated poisoning attacks

Innovation

• Introduces RAGForensics for traceback of poisoning attacks
• Uses iterative retrieval and LLM-guided poisoning detection
• Empirically effective against advanced poisoning attacks