Understanding and Evaluating Claw-like Agent Security Through a Computer-Systems Lens

2026-06-29

AI Summary

This study addresses a critical gap in AI agent safety evaluation by highlighting the overlooked cross-component attack surface, particularly for persistent, privileged agents resembling “Claw-like” systems whose compromise could entail severe consequences. Introducing a system-security perspective, the work models such agents as a three-layer architecture—comprising operating system, applications, and extensions—and proposes SafeClawArena, a novel adversarial evaluation framework encompassing 406 tasks with automated taint tracking across nine channels. Leveraging a containerized platform, canary credential tagging, and adversarial task generation, experiments reveal attack success rates up to 70%, with malicious plugins achieving 100% compromise. The proposed defense mechanism, SeClaw, effectively reduces the attack success rate against GPT-5.4 to 22%, while Claude-Opus-4.6 demonstrates robust resilience.

Abstract

Claw-like AI agents (e.g., OpenClaw) are always-on processes with persistent access to credentials, files, tools, and external services. They take on system-level responsibilities -- installing packages, maintaining state, scheduling subtasks, and mediating I/O -- making security failures far more severe than in other agents. Yet existing benchmarks focus on model responses and tool calls, leaving cross-component failure modes largely unmeasured. We adopt a computer-system analogy: treating a Claw-like agent as an agentic computer system whose gateway runtime plays an OS-like mediation role, whose Skills resemble user-installed applications, and whose Plugins resemble loadable extensions with runtime privileges. Each component has a classical counterpart whose protection mechanisms -- refined over decades of cybersecurity research -- are absent on the agent side. From this perspective, we develop SafeClawArena, a benchmark of 406 adversarial tasks across four attack surfaces (Skill Supply-Chain Integrity, Persistent State Exploitation, Cross-Boundary Data Flow, and Indirect Prompt Injection), executed in containerized replicas of real agent platforms with canary-marked credentials and evaluated via automated taint tracking across nine output channels. We evaluate three platforms (OpenClaw, NemoClaw, SeClaw) and five frontier LLMs. The highest attack success rate reaches 70%; malicious Plugins succeed in 100% of cases regardless of the LLM. SeClaw cuts GPT-5.4's attack success rate from 70% to 22%, partly through utility-security tradeoffs rather than active defenses, while Claude-Opus-4.6 already sits near a 22% floor on every platform. These results expose the inadequacy of current defenses and suggest directions for future hardening. Code and data: https://github.com/sunblaze-ucb/SafeClawArena.

Problem

• Claw-like agents
• agent security
• cross-component failure
• adversarial evaluation
• system-level responsibilities

Innovation

• Claw-like agents
• agent security
• adversarial benchmarking
• taint tracking
• computer-system analogy