EMPRA: Embedding Perturbation Rank Attack against Neural Ranking Models

2024-12-20

arXiv.org

AI Summary

This study addresses the vulnerability of neural ranking models to adversarial attacks in black-box settings. We propose an embedding-space directional perturbation method that enables cross-model generalizable attacks without requiring a surrogate model. Our approach introduces gradient-guided, imperceptible perturbations at the sentence-level embedding space, jointly optimized with semantic consistency constraints and query-feedback-driven iterative re-ranking to ensure adversarial texts remain semantically coherent and human-imperceptible. Experiments on MS MARCO V1 demonstrate that our method successfully promotes 96% of target documents—originally ranked between positions 51–100—to the top-10, substantially outperforming existing baselines. The method achieves high attack success rates, strong model-agnostic transferability, and superior textual naturalness, making it both effective and practical for real-world black-box ranking scenarios.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Recent research has shown that neural information retrieval techniques may be susceptible to adversarial attacks. Adversarial attacks seek to manipulate the ranking of documents, with the intention of exposing users to targeted content. In this paper, we introduce the Embedding Perturbation Rank Attack (EMPRA) method, a novel approach designed to perform adversarial attacks on black-box Neural Ranking Models (NRMs). EMPRA manipulates sentence-level embeddings, guiding them towards pertinent context related to the query while preserving semantic integrity. This process generates adversarial texts that seamlessly integrate with the original content and remain imperceptible to humans. Our extensive evaluation conducted on the widely-used MS MARCO V1 passage collection demonstrate the effectiveness of EMPRA against a wide range of state-of-the-art baselines in promoting a specific set of target documents within a given ranked results. Specifically, EMPRA successfully achieves a re-ranking of almost 96% of target documents originally ranked between 51-100 to rank within the top 10. Furthermore, EMPRA does not depend on surrogate models for adversarial text generation, enhancing its robustness against different NRMs in realistic settings.

Problem

• Adversarial attacks on neural ranking models
• Manipulating document rankings imperceptibly
• Enhancing target document promotion effectiveness

Innovation

• Embedding Perturbation Rank Attack
• Manipulates sentence-level embeddings
• Does not depend on surrogate models