🤖 AI Summary
Traditional red-teaming evaluations rely solely on binary attack success rates, overlooking the actual harm caused by an AI agent’s behavior after compromise. This work proposes a tool-call trajectory–based severity scale (L0–L6) that quantifies real-world risk along dimensions such as action reversibility, unauthorized impact on third parties, and privilege escalation. We introduce the first reusable, execution-trajectory–grounded severity assessment framework, combining a deterministic oracle with a panel of three state-of-the-art large language models to assign ordinal seven-level scores to unlabeled trajectories, achieving high inter-rater agreement (Krippendorff’s α = 0.91). Experiments on the AgentDojo benchmark uncover multiple security vulnerabilities masked by binary metrics and reveal systematic blind spots in models’ ability to recognize privilege escalation chains.
📝 Abstract
Agentic red-teaming benchmarks report whether an injected agent was compromised as a single bit: the attack succeeded, or it did not. We argue that this binary attack-success rate discards the information a defender most needs, namely how harmful the resulting action was. We introduce an action-graded harm rubric that scores an agent's tool-call trajectory on a seven-level ordinal scale (L0 to L6) according to whether the executed action was reversible, whether it crossed scope to reach another party, and whether it expanded privilege. We compute the scale two ways: a deterministic oracle that reads the trajectory and the attacker's stated goal, and a panel of three frontier language-model judges that read a tag-free account of the same trajectory. Across four victim models and two defenses on the AgentDojo workspace suite, severity grading exposes three cases the binary metric hides, including a defense that reports a zero attack-success rate while still permitting an externally visible cross-scope leak through an unfiltered tool. The judge panel reproduces the oracle with high ordinal agreement (Krippendorff's alpha = 0.91) but shares systematic blind spots that we characterize, most notably a failure to recognize escalation chains. Unlike prior work that provides harm taxonomies, harmful-task completion tests, execution-level safety benchmarks, or severity-aware simulation, our contribution is a reusable, trace-grounded severity instrument applied to the actual actions recorded in existing red-team logs. All code, prompts, and per-episode logs are released.