Detecting Contaminated Code-Generation Prompt Batches via Influence Functions

2026-08-14

AI Summary

This study addresses the vulnerability of code large language models to malicious prompts and the limitation of existing defenses that rely on known threat models. We propose CodeSIFT, a threat-model-agnostic detection paradigm that leverages influence functions to measure parameter space variations and employs statistical tests to identify anomalous prompt batches without requiring prior knowledge. This approach effectively overcomes the constraints of predefined vulnerability patterns, enabling the detection of unknown attacks. Experimental evaluations demonstrate that CodeSIFT achieves an AUROC of 0.98 across 3B to 7B parameter models with well-calibrated false positive rates. Furthermore, it significantly outperforms static analysis baselines, offering a robust and novel solution for enhancing the security of code generation systems against evolving adversarial threats.

Abstract

Large language models (LLMs) are increasingly used for code generation, yet they remain vulnerable to prompts that elicit insecure implementations. Existing defenses typically rely on predefined threat models or known vulnerability patterns, limiting their effectiveness against novel attacks. We propose CodeSIFT, a threat-model-agnostic detection method that leverages influence functions to identify batches of prompts that induce anomalous model behavior. Rather than detecting specific vulnerabilities, CodeSIFT measures the parameter-space influence of generated code and uses a statistical test to determine whether a candidate prompt set deviates from a benign reference distribution. To evaluate our approach, we introduce two benchmark datasets covering a variety of vulnerabilities. We evaluate CodeSIFT on three open-weight code LLMs ranging from 3B to 7B parameters, achieving AUROC scores of up to 0.98 at moderate-to-high injection rates, while maintaining well-calibrated false positive rates and substantially outperforming static analysis baselines. These results suggest that influence-function-based detection is a promising direction for identifying malicious code-generation prompts without requiring prior knowledge of the underlying attack class.

Problem

• Code Generation
• Prompt Injection
• Insecure Code
• Threat-model-agnostic Detection
• Large Language Models

Innovation

• Influence Functions
• Threat-model-agnostic
• CodeSIFT
• Parameter-space Influence
• Statistical Test