MazeRunner: Nonlinear Task and Clue Orchestration for LLM-driven Black-Box Automated Penetration Testing

2026-08-14

AI Summary

This study addresses the limitations of existing LLM-based penetration testing agents in black-box scenarios, specifically their tendency toward depth-first exploration and clue forgetting. To overcome these challenges, we propose a three-agent task-clue orchestration framework that decouples scheduling, execution, and review functions. By leveraging state persistence, this approach facilitates non-linear branch switching and long-range clue association, thereby transcending the constraints of linear workflows. Evaluations on the Hack The Box benchmark demonstrate that our method achieves a subtask completion rate of 47.7%. Furthermore, it significantly outperforms baselines in both root privilege acquisition and shell retrieval efficiency. These results confirm that the proposed framework effectively enhances attack surface coverage and execution efficacy in automated penetration testing.

Abstract

Penetration testing is essential yet resource-intensive. Although large language models (LLMs) show promise for automating security auditing, existing agents mainly execute end-to-end workflows in simplified linear scenarios. Real-world black-box testing is fundamentally nonlinear: the attack graph is initially unknown and must be incrementally inferred from environmental feedback. Observations may reveal multiple attack branches, failures are often ambiguous, and critical clues may span long action horizons. Existing agents therefore tend to become trapped in depth-first exploration, misdiagnose failures, and forget prior evidence. We present MazeRunner, an autonomous penetration testing system built on a three-agent task-and-clue orchestration framework. It separates global orchestration, context-intensive execution, and failure-oriented review while persistently maintaining task states and environmental evidence. This design supports action revision, prerequisite recovery, branch switching, and long-range clue correlation. We evaluate MazeRunner on 10 recently released HTB targets, limiting each system-target run to 20 million LLM tokens and preventing target-specific solution leakage. With Claude Sonnet 4.5, MazeRunner completes 47.7% of annotated subtasks, compared with 36.2% for PentestGPT-V2 and 34.2% for Claude Code. It achieves user-level or higher access on six targets, including root access on two; each same-model baseline reaches user-level access on only two targets and never obtains root access. Execution-trace analysis further shows that MazeRunner explores more attack branches and acquires shells more efficiently.

Problem

• Automated Penetration Testing
• Large Language Models
• Black-Box Testing
• Nonlinear Task Orchestration
• LLM Agents

Innovation

• Black-Box Penetration Testing
• Multi-Agent Orchestration
• Nonlinear Task Planning
• Long-range Clue Correlation
• Failure-oriented Review