Les Dissonances: Cross-Tool Harvesting and Polluting in Multi-Tool Empowered LLM Agents

2025-04-04

AI Summary

This work addresses security management challenges in multi-tool augmented LLM agents by conducting the first systematic analysis of task control-flow security, revealing a novel threat—Cross-Tool Harvesting and Poisoning (XTHP)—where attackers hijack control flow to exfiltrate or contaminate sensitive data across tools. We propose the first XTHP threat model, categorizing attacks into three vectors: hijacking, harvesting (XTH), and poisoning (XTP). To detect such vulnerabilities, we design Chord, the first dynamic security scanning framework for LLM workflows, integrating dynamic taint analysis, control-flow graph modeling, API behavior reverse inference, and automated vulnerability detection. Evaluation across 73 real-world tools shows that 80% are hijackable, 78% are vulnerable to XTH, and 41% exhibit XTP risks—demonstrating XTHP’s prevalence and severe security implications.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Multiagent Systems: Adversarial Agents

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large Language Model (LLM) agents are autonomous systems powered by LLMs, capable of reasoning and planning to solve problems by leveraging a set of tools. However, the integration of multi-tool capabilities in LLM agents introduces challenges in securely managing tools, ensuring their compatibility, handling dependency relationships, and protecting control flows within LLM agent workflows. In this paper, we present the first systematic security analysis of task control flows in multi-tool-enabled LLM agents. We identify a novel threat, Cross-Tool Harvesting and Polluting (XTHP), which includes multiple attack vectors to first hijack the normal control flows of agent tasks, and then collect and pollute confidential or private information within LLM agent systems. To understand the impact of this threat, we developed Chord, a dynamic scanning tool designed to automatically detect real-world agent tools susceptible to XTHP attacks. Our evaluation of 73 real-world tools from the repositories of two major LLM agent development frameworks, LangChain and LlamaIndex, revealed a significant security concern: 80% of the tools are vulnerable to hijacking attacks, 78% to XTH attacks, and 41% to XTP attacks, highlighting the prevalence of this threat.

Problem

• Security risks in multi-tool LLM agent workflows
• Cross-Tool Harvesting and Polluting (XTHP) threats
• Vulnerability of real-world tools to hijacking and data attacks

Innovation

• Dynamic scanning tool for XTHP detection
• Security analysis of multi-tool LLM agents
• Identifies Cross-Tool Harvesting and Polluting threats