A Systematic Security Evaluation of OpenClaw and Its Variants

2026-04-03

AI Summary

This work addresses the systemic safety risks introduced by tool-augmented AI agents—risks that often evade detection through conventional large language model evaluations. The authors present the first comprehensive benchmark comprising 205 test cases spanning the full agent lifecycle, systematically evaluating the safety of six OpenClaw frameworks across multiple backbone models. Through a unified evaluation protocol, formalized adversarial behavior modeling, and multidimensional risk design, the study reveals that safety vulnerabilities arise from the coupling of model capabilities, tool invocation, multi-step planning, and runtime orchestration. Empirical results demonstrate that all evaluated agents exhibit significant security flaws, with reconnaissance behaviors being the most prevalent. Distinct frameworks expose framework-specific high-severity risks—including credential leakage, lateral movement, and privilege escalation—and agents endowed with execution capabilities and persistent context tend to amplify localized weaknesses into system-wide failures.

Abstract

Tool-augmented AI agents substantially extend the practical capabilities of large language models, but they also introduce security risks that cannot be identified through model-only evaluation. In this paper, we present a systematic security assessment of six representative OpenClaw-series agent frameworks, namely OpenClaw, AutoClaw, QClaw, KimiClaw, MaxClaw, and ArkClaw, under multiple backbone models. To support this study, we construct a benchmark of 205 test cases covering representative attack behaviors across the full agent execution lifecycle, enabling unified evaluation of risk exposure at both the framework and model levels. Our results show that all evaluated agents exhibit substantial security vulnerabilities, and that agentized systems are significantly riskier than their underlying models used in isolation. In particular, reconnaissance and discovery behaviors emerge as the most common weaknesses, while different frameworks expose distinct high-risk profiles, including credential leakage, lateral movement, privilege escalation, and resource development. These findings indicate that the security of modern agent systems is shaped not only by the safety properties of the backbone model, but also by the coupling among model capability, tool use, multi-step planning, and runtime orchestration. We further show that once an agent is granted execution capability and persistent runtime context, weaknesses arising in early stages can be amplified into concrete system-level failures. Overall, our study highlights the need to move beyond prompt-level safeguards toward lifecycle-wide security governance for intelligent agent frameworks.

Problem

• AI agent security
• tool-augmented LLMs
• systematic security evaluation
• agent framework vulnerabilities
• lifecycle-wide risks

Innovation

• agent security
• tool-augmented LLMs
• systematic evaluation
• attack benchmark
• lifecycle governance