SHE: Trajectory-driven Safety Harness Evolution for LLM Agents

2026-08-10

AI Summary

Current safety mechanisms for large language model agents typically treat the safety control module as static and functionally coupled, limiting dynamic adaptation to emerging risks and hindering localized optimization. This work proposes SHE, a novel safety control module evolution framework that, for the first time, decouples the module into four well-defined components: system prompt, rule repository, safety memory, and tool policy. Guided by agent execution trajectories, SHE employs an attribution-driven evolutionary loop to enable structured diagnosis and refinement of safety boundaries. Evaluated on Agent-SafetyBench, the approach reduces attack success rates by 3.1× while improving utility on benign tasks. Moreover, the evolved modules demonstrate strong generalization to unseen threats on the AgentHarm benchmark and support effective cross-model transfer.

Abstract

The safety of large language model (LLM) agents depends not only on model weights but also on the agent harness that manages context, memory, tools, permissions, and runtime control. Existing safety mechanisms often treat the harness as a fixed deployment artifact, limiting their ability to evolve with emerging risks. Moreover, coupled functions across harness components obscure safety responsibility attribution, making localized evolution difficult. We propose Safety Harness Evolution (SHE), a framework that learns evolving safe boundaries from rollout trajectories. SHE decomposes the harness into four artifacts with explicit safety responsibilities, including the System Prompt, Rule Bank, Safety Memory, and Tool Policy, defining clear functional boundaries for localized evolution. Based on this decomposition, SHE introduces an attribution-guided evolution loop that converts trajectory failures into structured diagnoses, learns artifact-specific boundary refinements, and selects evolved harnesses through safety-utility validation. Experiments on Agent-SafetyBench demonstrate that SHE effectively enhances safety through harness evolution, achieving a 3.1x ASR reduction compared with static SafeHarness, while also improving benign utility. The evolved harness further generalizes to unseen risks on the held-out AgentHarm benchmark and transfers across agent models without additional evolution.

Problem

• LLM agents
• safety harness
• trajectory-driven evolution
• safety responsibility attribution
• dynamic safety adaptation

Innovation

• Safety Harness Evolution
• Trajectory-driven Learning
• Modular Safety Decomposition
• Attribution-guided Evolution
• LLM Agent Safety