Refusal Evaluation in Coding LLMs and Code Agents: A Systematic Review of Thirteen Malicious-Code Prompt Corpora (2023-2025)

2026-05-19

AI Summary

This study addresses the lack of systematic analysis of malicious code prompt corpora in evaluating the refusal capabilities of large language models, which has led to inconsistent methodologies, incomparable results, and fragmented classification schemes. For the first time, prompt corpora are treated as independent analytical units. Applying the PRISMA review protocol alongside a standardized data extraction template, the authors systematically examine the construction methods, prompt structures, reproducibility, and coverage of malicious categories across 13 publicly available corpora. Inter-annotator agreement is rigorously assessed using Fleiss’ kappa and bootstrap confidence intervals. The analysis reveals three critical methodological gaps: the absence of annotation benchmarks, cross-corpus incomparability, and inconsistent categorization. To address these issues, the study proposes standardized corpus development practices, including preregistration, multi-annotator validation, and a candidate universal taxonomy for malicious prompts.

Abstract

The evaluation of large language model refusal on malicious-coding tasks now spans at least thirteen publicly released prompt corpora (AdvBench, the CyberSecEval family, RMCBench, RedCode, MCGMark, JailbreakBench, CySecBench, MalwareBench, CIRCLE, MOCHA, ASTRA, Scam2Prompt / Innoc2Scam-bench, and JAWS-Bench), each constructed under a different protocol, released under different licensing terms, and validated (or not) against different inter-rater reliability standards. Existing surveys treat code security, jailbreak taxonomy, or vulnerability detection as the central object and mention these corpora only in passing. This paper reverses that framing: it treats the prompt datasets themselves as the unit of analysis. Following a PRISMA-style protocol, we specify a search strategy, screen the recent literature on coding-LLM refusal evaluation, apply a uniform extraction template to each in-scope corpus, and synthesize the resulting catalogue along construction methodology, prompt-construction taxonomy (modality, turn structure, elicitation style), reproducibility and licensing, and malware-category coverage. The synthesis surfaces three recurring methodological gaps: the absence of human-annotator baselines against which LLM-judge labels can be calibrated, the absence of cross-corpus comparability with refusal-rate statistics measuring non-equivalent constructs, and the fragmentation of malware-category taxonomies, with no canonical schema spanning the thirteen in-scope corpora. The review concludes with proposed methodological directions for next-generation corpora, including pre-registration of inclusion criteria, vendor-diverse multi-judge validation, Fleiss' kappa with bootstrap CI as the reliability baseline, and a candidate canonical taxonomy.

Problem

• refusal evaluation
• malicious-code prompts
• prompt corpora
• methodological gaps
• malware taxonomy

Innovation

• refusal evaluation
• malicious-code prompt corpora
• systematic review
• methodological gaps
• canonical taxonomy