LLM within MCP Matters: Measuring Inefficient Resource Utilization Driven by LLMs

2026-08-09

AI Summary

This study addresses the tendency of large language models (LLMs) to overlook reference data embedded in server instructions within Model Context Protocol (MCP) environments, instead inefficiently invoking search tools and wasting computational resources. Through 54,000 controlled trials, the authors systematically evaluate the behavior of 24 prominent LLMs on legal information retrieval tasks under MCP, employing tool ablation, a 2³ factorial design, and cross-model-family analysis. The work reveals— for the first time—that this behavior stems from preference rather than capability deficits. Notably, removing the search tool yields over 98% accuracy in 23 out of 24 models, and combining three targeted prompting interventions restores accuracy above 86% for 20 out of 24 models even when the tool is present. The findings advocate for MCP hosts to explicitly prioritize server-provided instructions.

Abstract

The Model Context Protocol (MCP) standardizes how servers expose data and tools to Large Language Models (LLMs). A common server design embeds frequently used reference data, such as identifier lookup tables, directly in the server instructions: the system-prompt text a server hands to the host application. When a query concerns an entry of the embedded table, the model can act on it immediately instead of re-discovering the same information through a search tool. We test whether client LLMs actually consume such instruction-embedded data, reporting a 54,000-trial study across 24 LLMs (9 Claude, 6 Gemini, 9 GPT) on a production legal-information MCP server. A diagnostic condition that removes the competing search tool shows that failures are dominated by behavioral preference rather than missing capability. With search unavailable, 23 of 24 models read the embedded data reliably (hit ratio at least 98%); with a search tool merely present, 9 models drop below 15%. A 2^3 factorial analysis of three instruction-level interventions reveals strong interaction effects: combining all three restores at least 86% for 20 of 24 models, but individual interventions can backfire for specific model families. Per-server prompt engineering is therefore a workaround rather than a fix; we argue that MCP host applications should provide an explicit mechanism that places server instructions ahead of tool selection in the client LLM's deliberation.

Problem

• Large Language Models
• Model Context Protocol
• Inefficient Resource Utilization
• System Prompt
• Tool Usage

Innovation

• Model Context Protocol
• instruction-following efficiency
• tool overuse
• prompt engineering
• LLM behavioral bias