Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generation

2025-09-17

AI Summary

RAG systems are vulnerable to knowledge base poisoning attacks, yet existing methods struggle to precisely localize malicious text responsible for erroneous outputs. To address this, we propose the first black-box attribution framework that operates without access to model internals. Our approach jointly models retrieval ranking bias, semantic relevance anomalies, and generative output perturbations to dynamically delineate the attribution scope for individual hallucination events; it further isolates poisoned content via unsupervised clustering. The framework demonstrates robustness under adaptive attacks and multi-attacker settings. Extensive experiments across seven datasets and fifteen distinct poisoning attack types show that our method significantly outperforms state-of-the-art baselines, achieving high-precision identification of contaminated knowledge entries. By providing interpretable, fine-grained attribution diagnostics, our work advances the security and accountability of RAG systems.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Retrieval-Augmented Generation (RAG) integrates external knowledge into large language models to improve response quality. However, recent work has shown that RAG systems are highly vulnerable to poisoning attacks, where malicious texts are inserted into the knowledge database to influence model outputs. While several defenses have been proposed, they are often circumvented by more adaptive or sophisticated attacks. This paper presents RAGOrigin, a black-box responsibility attribution framework designed to identify which texts in the knowledge database are responsible for misleading or incorrect generations. Our method constructs a focused attribution scope tailored to each misgeneration event and assigns a responsibility score to each candidate text by evaluating its retrieval ranking, semantic relevance, and influence on the generated response. The system then isolates poisoned texts using an unsupervised clustering method. We evaluate RAGOrigin across seven datasets and fifteen poisoning attacks, including newly developed adaptive poisoning strategies and multi-attacker scenarios. Our approach outperforms existing baselines in identifying poisoned content and remains robust under dynamic and noisy conditions. These results suggest that RAGOrigin provides a practical and effective solution for tracing the origins of corrupted knowledge in RAG systems.

Problem

• Identifying poisoned texts causing incorrect outputs in RAG systems
• Attributing responsibility for misleading generations to database sources
• Detecting malicious knowledge injections through unsupervised clustering analysis

Innovation

• Black-box responsibility attribution framework
• Unsupervised clustering isolates poisoned texts
• Evaluates retrieval ranking and semantic relevance