Mastermind: Strategy-grounded Learning for Repository-Scale Vulnerability Reproduction

2026-07-02

AI Summary

This work addresses the challenge of large language model (LLM) agents failing to reproduce warehouse-scale vulnerabilities due to suboptimal strategy selection. To overcome this, the authors propose a dual-loop framework that decouples transferable high-level strategy learning from task-specific execution. For the first time, strategies serve as the fundamental unit of learning: a trainable planner learns generalizable reproduction strategies and guides multiple frozen executors to accomplish tasks, enabling cross-task strategy reuse. The planner is trained via supervised fine-tuning (SFT) combined with milestone-based GRPO and augmented with an experience replay buffer. Evaluated on the CyberGym benchmark, the approach achieves an 84.5% pass rate with a GPT-5.5 executor, significantly outperforming baselines and also enhancing the performance of GPT-5.4 mini and GLM-5.1.

Abstract

Repository-level vulnerability reproduction is a demanding software engineering (SE) task: an agent must inspect a codebase, infer the input grammar that reaches a vulnerable path, construct a proof-of-conceptv(PoC), and verify that the crash disappears on the patched build. Recent LLM agents can often execute these steps when the approach is correct, yet they still fail by choosing the wrong strategy. This paper argues that strategy, rather than the full action trajectory, is the right learning unit for such SE agents: it is compact enough to optimize, concrete enough to guide execution, and stable enough to store and reuse across attempts. We present Mastermind, a dual-loop framework that separates transferable strategy learning from task-specific experience. A trainable planner learns reusable vulnerability-reproduction strategies through SFT and milestone-based GRPO, while an experience loop maintains task-local strategy records that guide subsequent attempts. The planner is trained independently of the executor, allowing strategy learning to improve multiple frozen executors without modifying their action-generation capability. We evaluate Mastermind on CyberGym using 260 training tasks and 200 held-out evaluation tasks. With GPT-5.5 as the frozen executor, Mastermind achieves an 84.5% pass rate, outperforming open-book PoC context (60.0%), Best-of-8 sampling (63.0%), and iterative improvement (77.0%). The same planner also improves GPT-5.4 mini and GLM~5.1 from 45.0% and 58.5% to 60.0% and 71.0%. These results demonstrate that learning high-level strategies is an effective and transferable mechanism for improving repository-scale SE agents.

Problem

• vulnerability reproduction
• repository-scale
• strategy learning
• software engineering agents
• large language models

Innovation

• strategy-grounded learning
• dual-loop framework
• vulnerability reproduction
• transferable strategy
• LLM agent