MCPSecBench: A Systematic Security Benchmark and Playground for Testing Model Context Protocols

2025-08-17

AI Summary

The Model Context Protocol (MCP), an open standard enabling LLM agents to interact with external tools, significantly expands the attack surface but lacks a systematic security evaluation framework. Method: We propose the first MCP security taxonomy—covering four attack surfaces (client, server, transport protocol, and tool invocation)—with 17 fine-grained attack categories. We design a modular, extensible security benchmark and experimental platform supporting automated, cross-vendor (e.g., Claude, OpenAI, Cursor) multi-dimensional evaluation, and introduce a unified testing framework spanning prompt-level and tool-call-level vulnerabilities. Contribution/Results: Empirical evaluation reveals that over 85% of identified attacks succeed on at least one mainstream MCP platform; critical vulnerabilities are pervasive, and security capabilities vary markedly across vendors. This work establishes the first standardized security assessment paradigm for the MCP ecosystem.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Adversarial Agents

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large Language Models (LLMs) are increasingly integrated into real-world applications via the Model Context Protocol (MCP), a universal, open standard for connecting AI agents with data sources and external tools. While MCP enhances the capabilities of LLM-based agents, it also introduces new security risks and expands their attack surfaces. In this paper, we present the first systematic taxonomy of MCP security, identifying 17 attack types across 4 primary attack surfaces. We introduce MCPSecBench, a comprehensive security benchmark and playground that integrates prompt datasets, MCP servers, MCP clients, and attack scripts to evaluate these attacks across three major MCP providers. Our benchmark is modular and extensible, allowing researchers to incorporate custom implementations of clients, servers, and transport protocols for systematic security assessment. Experimental results show that over 85% of the identified attacks successfully compromise at least one platform, with core vulnerabilities universally affecting Claude, OpenAI, and Cursor, while prompt-based and tool-centric attacks exhibit considerable variability across different hosts and models. Overall, MCPSecBench standardizes the evaluation of MCP security and enables rigorous testing across all MCP layers.

Problem

• Systematically identifies security risks in Model Context Protocols
• Evaluates attack vulnerabilities across major MCP platforms
• Provides benchmark for testing MCP security implementations

Innovation

• Systematic security taxonomy for MCP
• Modular benchmark with attack evaluation
• Testing framework for MCP vulnerabilities