Program Analysis Guided LLM Agent for Proof-of-Concept Generation

2026-04-08

AI Summary

Existing approaches to automatically generating reliable proof-of-concept (PoC) inputs that trigger vulnerabilities suffer from either low success rates or poor scalability. This work proposes a novel LLM-based agent framework that integrates lightweight static analysis with dynamic execution feedback—including sanitizer instrumentation and coverage tracking—to guide the generation of more effective PoCs. For the first time, it deeply incorporates rule-driven static analysis and dynamic runtime information into large language models. Experimental results demonstrate that the proposed method achieves a 132% improvement in success rate over the current state-of-the-art agent-based approach on PoC generation tasks, significantly enhancing both effectiveness and scalability.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Software developers frequently receive vulnerability reports that require them to reproduce the vulnerability in a reliable manner by generating a proof-of-concept (PoC) input that triggers it. Given the source code for a software project and a specific code location for a potential vulnerability, automatically generating a PoC for the given vulnerability has been a challenging research problem. Symbolic execution and fuzzing techniques require expert guidance and manual steps and face scalability challenges for PoC generation. Although recent advances in LLMs have increased the level of automation and scalability, the success rate of PoC generation with LLMs remains quite low. In this paper, we present a novel approach called Program Analysis Guided proof of concept generation agENT (PAGENT) that is scalable and significantly improves the success rate of automated PoC generation compared to prior results. PAGENT integrates lightweight and rule-based static analysis phases for providing static analysis guidance and sanitizer-based profiling and coverage information for providing dynamic analysis guidance with a PoC generation agent. Our experiments demonstrate that the resulting hybrid approach significantly outperforms the prior top-performing agentic approach by 132% for the PoC generation task.

Problem

• Proof-of-Concept generation
• vulnerability reproduction
• program analysis
• automated PoC
• software security

Innovation

• Program Analysis
• Proof-of-Concept Generation
• LLM Agent
• Static and Dynamic Analysis
• Vulnerability Reproduction