AdversarialCoT: Single-Document Retrieval Poisoning for LLM Reasoning

2026-04-13

AI Summary

This work addresses the security threat posed by knowledge base poisoning attacks in Retrieval-Augmented Generation (RAG) systems and proposes AdversarialCoT, a novel method that, for the first time, enables query-specific attacks using only a single malicious document. By extracting the reasoning structure of the target large language model, AdversarialCoT constructs and iteratively optimizes an adversarial chain-of-thought, leveraging the retrieval-generation feedback loop to steer the model toward erroneous reasoning. Experimental results demonstrate that a single adversarial document substantially degrades reasoning accuracy across mainstream large language models, overcoming the prior requirement for large-scale data poisoning and precisely exposing a critical vulnerability in the reasoning robustness of RAG systems.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Retrieval-augmented generation (RAG) enhances large language model (LLM) reasoning by retrieving external documents, but also opens up new attack surfaces. We study knowledge-base poisoning attacks in RAG, where an attacker injects malicious content into the retrieval corpus, which is then naturally surfaced by the retriever and consumed by the LLM during reasoning. Unlike prior work that floods the corpus with poisoned documents, we propose AdversarialCoT, a query-specific attack that poisons only a single document in the corpus. AdversarialCoT first extracts the target LLM's reasoning framework to guide the construction of an initial adversarial chain-of-thought (CoT). The adversarial document is iteratively refined through interactions with the LLM, progressively exposing and exploiting critical reasoning vulnerabilities. Experiments on benchmark LLMs show that a single adversarial document can significantly degrade reasoning accuracy, revealing subtle yet impactful weaknesses. This study exposes security risks in RAG systems and provides actionable insights for designing more robust LLM reasoning pipelines.

Problem

• retrieval-augmented generation
• knowledge-base poisoning
• adversarial attack
• large language model
• reasoning vulnerability

Innovation

• AdversarialCoT
• retrieval-augmented generation
• knowledge-base poisoning
• chain-of-thought
• LLM reasoning