Gaming Tool Preferences in Agentic LLMs

2025-05-23

AI Summary

This work exposes a critical vulnerability in large language models (LLMs): their tool invocation behavior is highly sensitive to textual tool descriptions—minor, semantically preserving edits can induce drastic shifts in model preference, with invocation rates increasing over 10× in some cases. We systematically identify and name this phenomenon “tool preference manipulation.” To rigorously characterize it, we design a controllable evaluation framework grounded in the Model Context Protocol (MCP) and conduct cross-model experiments across ten state-of-the-art models—including GPT-4.1 and Qwen2.5-7B—quantifying the impact of diverse description-editing strategies. Our results reveal both the prevalence and model-specific generalization patterns of this phenomenon, demonstrating that susceptibility varies significantly across architectures and scales. This provides the first empirical foundation for diagnosing and improving tool selection mechanisms, advancing the development of more robust, reliable, and trustworthy agent-based tool invocation systems.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Learning Human Values and Preferences

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models (LLMs) can now access a wide range of external tools, thanks to the Model Context Protocol (MCP). This greatly expands their abilities as various agents. However, LLMs rely entirely on the text descriptions of tools to decide which ones to use--a process that is surprisingly fragile. In this work, we expose a vulnerability in prevalent tool/function-calling protocols by investigating a series of edits to tool descriptions, some of which can drastically increase a tool's usage from LLMs when competing with alternatives. Through controlled experiments, we show that tools with properly edited descriptions receive over 10 times more usage from GPT-4.1 and Qwen2.5-7B than tools with original descriptions. We further evaluate how various edits to tool descriptions perform when competing directly with one another and how these trends generalize or differ across a broader set of 10 different models. These phenomenons, while giving developers a powerful way to promote their tools, underscore the need for a more reliable foundation for agentic LLMs to select and utilize tools and resources.

Problem

• LLMs rely on fragile text descriptions for tool selection
• Edited tool descriptions drastically increase usage by LLMs
• Need reliable foundation for agentic LLMs to choose tools

Innovation

• Utilizes Model Context Protocol (MCP)
• Edits tool descriptions for increased usage
• Tests across 10 different LLM models