Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models

2024-08-15

AI Summary

Existing evaluations of large language models (LLMs) in cybersecurity lack systematic, quantitative assessment—particularly for penetration testing. Method: We propose the first open-source benchmark framework specifically designed to quantify LLMs’ penetration testing capabilities. It comprises 40 real-world CTF challenges with fine-grained subtasks, a sandboxed command-execution environment, a multi-agent scaffolding architecture (including structured bash, action-only, pseudoterminal, and web-search agents), and a cross-model consistency evaluation protocol. Crucially, we introduce a subtask decomposition mechanism enabling progressive, granular diagnosis of LM agent capabilities and achieve, for the first time, fully automated and observable end-to-end penetration testing evaluation. Results: Experiments show GPT-4o and Claude 3.5 Sonnet solve human-level tasks—requiring ~11 minutes for humans—in zero-shot settings; the hardest tasks take humans over 24 hours. All code and data are publicly released.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Language Model (LM) agents for cybersecurity that are capable of autonomously identifying vulnerabilities and executing exploits have potential to cause real-world impact. Policymakers, model providers, and researchers in the AI and cybersecurity communities are interested in quantifying the capabilities of such agents to help mitigate cyberrisk and investigate opportunities for penetration testing. Toward that end, we introduce Cybench, a framework for specifying cybersecurity tasks and evaluating agents on those tasks. We include 40 professional-level Capture the Flag (CTF) tasks from 4 distinct CTF competitions, chosen to be recent, meaningful, and spanning a wide range of difficulties. Each task includes its own description, starter files, and is initialized in an environment where an agent can execute commands and observe outputs. Since many tasks are beyond the capabilities of existing LM agents, we introduce subtasks for each task, which break down a task into intermediary steps for a more detailed evaluation. To evaluate agent capabilities, we construct a cybersecurity agent and evaluate 8 models: GPT-4o, OpenAI o1-preview, Claude 3 Opus, Claude 3.5 Sonnet, Mixtral 8x22b Instruct, Gemini 1.5 Pro, Llama 3 70B Chat, and Llama 3.1 405B Instruct. For the top performing models (GPT-4o and Claude 3.5 Sonnet), we further investigate performance across 4 agent scaffolds (structed bash, action-only, pseudoterminal, and web search). Without subtask guidance, agents leveraging Claude 3.5 Sonnet, GPT-4o, OpenAI o1-preview, and Claude 3 Opus successfully solved complete tasks that took human teams up to 11 minutes to solve. In comparison, the most difficult task took human teams 24 hours and 54 minutes to solve. All code and data are publicly available at https://cybench.github.io.

Problem

• Evaluating cybersecurity capabilities of Language Model agents
• Mitigating cyberrisks through autonomous vulnerability identification
• Assessing LM agents in professional-level Capture the Flag tasks

Innovation

• Framework for evaluating LM cybersecurity capabilities
• Includes 40 professional Capture the Flag tasks
• Introduces subtasks for detailed agent evaluation