A Systematic Literature Review on Automated Exploit and Security Test Generation

2025-02-07

AI Summary

High-quality exploits for validating vulnerability repair tools are scarce, costly to construct manually, and require specialized expertise. Method: This study conducts the first systematic literature review (SLR) on exploit generation techniques, analyzing 128 publications to map state-of-the-art approaches for C/C++ memory-safety vulnerabilities and PHP/Java web injection vulnerabilities. We propose a novel, structured taxonomy categorizing methods into four classes—automatic exploit generation, security testing, fuzzing-based exploitation, and hybrid techniques—and perform the first empirical usability assessment across open-source availability, reproducibility, and industrial deployability. Contribution/Results: Our analysis uncovers critical bottlenecks, including low open-source adoption and poor industrial integration. The work delivers a reusable research framework and evidence-based tool selection guidelines, significantly enhancing the feasibility and efficiency of automated vulnerability repair validation and testing.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Computer Vision: Adversarial Attacks & Robustness
• Constraint Satisfaction and Optimization: Solvers and Tools

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements

Abstract

The exploit or the Proof of Concept of the vulnerability plays an important role in developing superior vulnerability repair techniques, as it can be used as an oracle to verify the correctness of the patches generated by the tools. However, the vulnerability exploits are often unavailable and require time and expert knowledge to craft. Obtaining them from the exploit generation techniques is another potential solution. The goal of this survey is to aid the researchers and practitioners in understanding the existing techniques for exploit generation through the analysis of their characteristics and their usability in practice. We identify a list of exploit generation techniques from literature and group them into four categories: automated exploit generation, security testing, fuzzing, and other techniques. Most of the techniques focus on the memory-based vulnerabilities in C/C++ programs and web-based injection vulnerabilities in PHP and Java applications. We found only a few studies that publicly provided usable tools associated with their techniques.

Problem

• Exploit generation techniques survey
• Automated vulnerability verification methods
• Security testing in C/C++ and PHP/Java

Innovation

• Automated exploit generation techniques
• Security testing and fuzzing methods
• Focus on C/C++ and web vulnerabilities