Cross-Cutting Security Analysis of LLM-Generated Code via Metamorphic Testing and Association Rule Mining

2026-07-13

AI Summary

This work addresses the challenge that code generated by large language models (LLMs) often contains composite vulnerabilities spanning multiple security dimensions, which evade detection by conventional methods focused on isolated defects. The authors propose a systematic approach integrating security-oriented metamorphic relations, association rule mining, and prompt-level risk analysis to uncover structural co-occurrence patterns of vulnerabilities in LLM-generated code and establish an interpretable mapping from these vulnerabilities back to risky prompt characteristics. Analyzing 3,700 code samples, they find that 68.8% contain security flaws; high-confidence co-occurrence rules emerge—such as cross-site scripting (XSS) combined with weak encryption indicating hard-coded credentials—and demonstrate that 65.5% of prompts consistently trigger unsafe outputs across multiple models, thereby motivating cluster-aware security validation.

Abstract

Large language models (LLMs) frequently generate code with security vulnerabilities, yet these weaknesses are rarely isolated: they often span multiple concern areas simultaneously, reflecting the cross-cutting nature of security in software. We present a framework that combines security-oriented Metamorphic Relations (MRs) with Association Rule (AR) mining to detect vulnerabilities in LLM-generated code, uncover their co-violation structure, and trace that structure back to prompt-level risk factors. We define nine MRs covering major CWE categories, including SQL injection, XSS, command injection, path traversal, hard-coded credentials, weak cryptography, and memory-safety errors, and apply them using an LLM-based judge to 3,700 code snippets generated by five open models from the LLMSecEval benchmark. The results show that 68.8% of snippets violate at least one MR, with hard-coded credentials (79.1%) and command injection (74.4%) among the most prevalent applicable failures. AR mining reveals strong cross-cutting co-violation patterns, notably that XSS and weak cryptography co-violations predict hard-coded credentials with 82.5% confidence (lift = 3.23), along with tightly coupled clusters linking authentication, credential handling, and cryptographic weakness, as well as input-handling and memory-safety failures. We then perform prompt-level risk analysis and find that database- and authentication-related prompts are strong predictors of broad cross-cutting insecurity, while 65.5% of prompts yield consistent violation outcomes across all five models. These findings show that insecure code generation is not merely a collection of independent defects, but a structured and prompt-conditioned phenomenon, motivating cluster-aware verification and prompt-level intervention for safer LLM-assisted programming.

Problem

• LLM-generated code
• security vulnerabilities
• cross-cutting concerns
• metamorphic testing
• association rule mining

Innovation

• Metamorphic Testing
• Association Rule Mining
• LLM-Generated Code
• Cross-Cutting Security
• Prompt-Level Risk Analysis