REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems

2026-08-11

AI Summary

Current safety evaluations of large language model (LLM) agents predominantly rely on single-metric attack success rates, which inadequately capture the real-world risk of policy violations during environmental interaction. This work proposes an executable red-teaming framework that generates attacks grounded in explicit safety constraints, executes them within an isolated sandbox, and validates actual harm through service credentials and final-state changes. The study introduces a novel state-anchored diagnostic mechanism to uncover the agent’s “recognition–execution gap” and designs a training-free policy reminder that substantially reduces policy violations. Evaluated across 1,661 test cases involving six models and three agent frameworks, the macro-average attack success rate reaches 65.69%; notably, the policy reminder reduces confirmed violation rates by over 70 percentage points.

Abstract

Large language model (LLM) agents combine language-based reasoning with external tools to perform complex tasks. Adversarial inputs can exploit interactions between the agent and its environment, causing the agent to violate safety policies during execution. Yet existing evaluations often reduce agent safety to a single attack success rate (ASR), collapsing exposure, execution, observation, and adjudication and potentially conflating actual violations with evidence visibility. We introduce REDAgentBench, an executable framework for autonomous red-teaming and faithful measurement. It derives attacks from explicit safety constraints and associated agent-system vulnerabilities, runs them in isolated service sandboxes, and verifies harmful effects from service receipts and final-state changes. The benchmark contains 1,661 cases across five service surfaces. Across six models and three agent harnesses, macro-average ASR is 65.69%; reported ASR varies with harness and evidence view, while evaluation-context disclosure changes execution behavior. In a state-grounded diagnostic cohort, almost one in five confirmed violations with resolved action anchors occurs after the agent states the relevant constraint or risk, revealing a Recognition--Execution Gap. Finally, a training-free policy reminder reduces confirmed violations by more than 70 percentage points in matched replay. These findings show that executable evaluation can improve safety measurement and identify actionable intervention points.

Problem

• LLM agent safety
• adversarial evaluation
• attack success rate
• executable red teaming
• safety measurement

Innovation

• executable red teaming
• faithful safety measurement
• LLM agent evaluation
• Recognition-Execution Gap
• policy reminder