DSPrompt: Dynamic Soft Prompt Defense Against M-RAG Corruption

2026-08-17

AI Summary

This study addresses the vulnerability of multimodal retrieval-augmented generation systems to data poisoning attacks by proposing a dynamic soft prompt defense framework. The method leverages learnable soft prompts to reshape retriever embedding semantics, integrating adaptive length scheduling with dynamic minimax game training to achieve efficient defense while keeping encoders frozen. Crucially, this approach requires no modifications to the retrieval pipeline and introduces no additional inference overhead. Experimental results demonstrate that the framework significantly reduces attack success rates while preserving retrieval utility, with computational costs substantially lower than existing baselines. Consequently, this work offers a lightweight yet effective solution for enhancing the security of multimodal systems against adversarial threats without compromising operational efficiency.

Abstract

Multimodal Retrieval Augmented Generation (M-RAG) is increasingly vulnerable to adversarial attacks where malicious data are crafted to produce embeddings that align with benign entries in the vector space, deceiving retrieval and inducing harmful outputs. Existing defenses primarily operate at query time, relying on auxiliary detectors, similarity re-ranking, or feature-consistency checks. However, these approaches suffer from non-trivial inference overhead, generalize poorly to unseen attack strategies, and often assume specific attack distributions. To address this, we propose DSPrompt, a Dynamic Soft Prompt defense framework that directly reshapes the retriever's embedding semantics, without modifying the retrieval pipeline. It inserts few learnable soft prompts into each layer of the visual and textual encoders of a frozen retriever, utilizing a shallow-to-deep length schedule that is adaptive to the capacity in the model layers. These prompts are trained under a dynamic min-max scheme: an online multimodal attacker continually crafts hard adversarial documents against the current retriever, while the defender is updated to push such documents out of the top-k while preserving the ranking and diversity of benign evidence. Because the defended encoder can be pre-computed and indexed exactly as in standard dense retrieval, DSPrompt incurs no additional per-query optimization and introduces fewer than 1% additional parameters. Extensive experiments across four benchmarks and three representative poisoning attacks show that DSPrompt substantially reduces the attack success rate and poison retrieval rate while maintaining near-lossless retrieval utility and generation fidelity, consistently outperforming existing defense baselines at a fraction of their computational cost.

Problem

• Multimodal Retrieval Augmented Generation
• Adversarial Attacks
• Defense
• Retrieval Corruption

Innovation

• Dynamic Soft Prompt
• M-RAG Defense
• Min-Max Training
• Embedding Semantics Reshaping
• Zero Inference Overhead