MT4DP: Data Poisoning Attack Detection for DL-based Code Search Models via Metamorphic Testing

2025-07-15

AI Summary

Data poisoning attacks pose a severe threat to the security of deep learning–based code search models, yet existing detection methods exhibit limited effectiveness. To address this, we propose MT4DP, a mutation-testing–based detection framework. Its core innovation lies in constructing semantic-equivalent mutation relations (SE-MRs), leveraging high-frequency tokens as anchors to generate semantically equivalent queries and identifying model anomalies via ranking consistency analysis. MT4DP integrates mutation testing, semantic similarity modeling, and ranking-difference detection—requiring no access to training data or labels, and relying solely on model inference behavior. Extensive experiments across multiple state-of-the-art code search models demonstrate that MT4DP achieves an average 191% improvement in F1-score and a 265% gain in precision over the best baseline, significantly enhancing robustness against stealthy data poisoning attacks.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web search models and ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Recently, several studies have indicated that data poisoning attacks pose a severe security threat to deep learning-based (DL-based) code search models. Attackers inject carefully crafted malicious patterns into the training data, misleading the code search model to learn these patterns during training. During the usage of the poisoned code search model for inference, once the malicious pattern is triggered, the model tends to rank the vulnerability code higher. However, existing detection methods for data poisoning attacks on DL-based code search models remain insufficiently effective. To address this critical security issue, we propose MT4DP, a Data Poisoning Attack Detection Framework for DL-based Code Search Models via Metamorphic Testing. MT4DP introduces a novel Semantically Equivalent Metamorphic Relation (SE-MR) designed to detect data poisoning attacks on DL-based code search models. Specifically, MT4DP first identifies the high-frequency words from search queries as potential poisoning targets and takes their corresponding queries as the source queries. For each source query, MT4DP generates two semantically equivalent follow-up queries and retrieves its source ranking list. Then, each source ranking list is re-ranked based on the semantic similarities between its code snippets and the follow-up queries. Finally, variances between the source and re-ranked lists are calculated to reveal violations of the SE-MR and warn the data poisoning attack. Experimental results demonstrate that MT4DP significantly enhances the detection of data poisoning attacks on DL-based code search models, outperforming the best baseline by 191% on average F1 score and 265% on average precision. Our work aims to promote further research into effective techniques for mitigating data poisoning threats on DL-based code search models.

Problem

• Detects data poisoning in DL-based code search models
• Identifies malicious patterns in training data
• Improves detection effectiveness via metamorphic testing

Innovation

• Uses metamorphic testing for poisoning detection
• Introduces Semantically Equivalent Metamorphic Relation
• Generates follow-up queries to reveal attacks