Synthesizing Multi-Agent Harnesses for Vulnerability Discovery

2026-04-22

AI Summary

This work addresses the inefficiency of existing handcrafted multi-agent collaboration frameworks (harnesses) in vulnerability discovery and the limitations of current automated optimization approaches, which suffer from restricted search spaces and coarse-grained diagnostics. To overcome these challenges, the authors propose AgentFlow, the first framework to formalize the harness design space as a searchable, typed graph structure. AgentFlow employs a domain-specific language to uniformly model agent roles, prompts, tools, communication topologies, and coordination protocols, and dynamically rewrites the harness using runtime program feedback. It further introduces an instrumentation-based, fine-grained diagnostic mechanism that drives an outer-loop optimizer for precise iterative refinement. Experimental results demonstrate that AgentFlow achieves a state-of-the-art score of 84.3% on TerminalBench-2 and discovers ten previously unknown zero-day vulnerabilities in Google Chrome, including two high-severity sandbox escape flaws (CVE-2026-5280 and CVE-2026-6297).

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Multiagent Systems: Other Foundations of Multi Agent Systems
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

LLM agents have begun to find real security vulnerabilities that human auditors and automated fuzzers missed for decades, in source-available targets where the analyst can build and instrument the code. In practice the work is split among several agents, wired together by a harness: the program that fixes which roles exist, how they pass information, which tools each may call, and how retries are coordinated. When the language model is held fixed, changing only the harness can still change success rates by several-fold on public agent benchmarks, yet most harnesses are written by hand; recent harness optimizers each search only a narrow slice of the design space and rely on coarse pass/fail feedback that gives no diagnostic signal about why a trial failed. AgentFlow addresses both limitations with a typed graph DSL whose search space jointly covers agent roles, prompts, tools, communication topology, and coordination protocol, paired with a feedback-driven outer loop that reads runtime signals from the target program itself to diagnose which part of the harness caused the failure and rewrite it accordingly. We evaluate AgentFlow on TerminalBench-2 with Claude Opus 4.6 and on Google Chrome with Kimi K2.5. AgentFlow reaches 84.3% on TerminalBench-2, the highest score in the public leaderboard snapshot we evaluate against, and discovers ten previously unknown zero-day vulnerabilities in Google Chrome, including two Critical sandbox-escape vulnerabilities (CVE-2026-5280 and CVE-2026-6297).

Problem

• multi-agent harness
• vulnerability discovery
• harness optimization
• LLM agents
• feedback-driven diagnosis

Innovation

• AgentFlow
• multi-agent harness
• vulnerability discovery
• typed graph DSL
• feedback-driven optimization