Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verification

2026-07-02

AI Summary

Current safety testing of large language model agents relies on expert-crafted static rules, which are difficult to scale and maintain. To address this limitation, this work proposes Vera, a framework that systematically adapts software engineering testing principles to nondeterministic agents, establishing an end-to-end, scalable safety evaluation paradigm. Vera structures domain knowledge through a risk taxonomy, synthesizes executable test cases via cross-dimensional composition, and employs controllable agents within isolated sandboxes to enable multi-turn interactions. Safety violations are adaptively verified using environmental states and tool invocation evidence. Evaluated across four mainstream agent frameworks, Vera achieves an average attack success rate of 93.9% and introduces Vera-Bench—a benchmark comprising 1,600 test cases spanning 124 risk categories and three execution environments.

Abstract

LLM agents increasingly perform autonomous actions through external tools, leading to complex and evolving safety risks. However, existing safety testing targets expert-designed safety violations, and the corresponding outcomes are evaluated by hard-coded rules, making them costly to extend as agents evolve. To this end, we present Vera, an end-to-end automated safety testing framework that instantiates software engineering testing principles for non-deterministic agents through a three-stage, self-reinforcing pipeline. First, a literature-driven exploration continuously discovers and structures emerging risks into taxonomies of safety risks, attack methods, and tool execution environments. Second, combinatorial composition across taxonomy dimensions produces executable safety cases, each specifying a concrete safety goal, a programmatically constructed initial state, and a deterministic verification predicate grounded in observable artifacts. Third, adaptive execution runs heterogeneous agents in isolated sandboxes where a control agent steers multi-turn interaction based on runtime observations, while evidence-grounded verifiers judge outcomes from environment state and tool-call evidence rather than model self-report. We evaluate Vera on four production agent frameworks (OpenClaw, Hermes, Codex, Claude Code), revealing substantial safety weaknesses, with average attack success rates reaching 93.9\% under multi-channel attacks; we also release Vera-Bench, comprising 1600 executable safety cases spanning 124 risk categories across three execution settings. These results indicate that modular, executable testing infrastructure is essential for rigorous and maintainable safety evaluation of rapidly evolving agentic systems at scale. The code is publicly available at https://github.com/Yunhao-Feng/Vera.

Problem

• LLM agents
• safety testing
• risk discovery
• automated verification
• scalable evaluation

Innovation

• automated safety testing
• LLM agents
• evidence-grounded verification
• combinatorial test generation
• risk taxonomy