Tool-Planner: Task Planning with Clusters across Multiple Tools

2024-06-06

AI Summary

To address the instability in planning and redundant error correction during multi-tool invocation by large language models (LLMs), this paper proposes a toolkit framework based on functional clustering. The method automatically groups tools into high-level semantic abstractions—toolkits—thereby elevating planning granularity from individual tools to toolkit-level units. It introduces a hierarchical prompting mechanism and a toolkit-aware planning-execution-feedback loop, enabling semantically consistent fault-tolerant re-planning. This design significantly reduces error-correction overhead while enhancing planning robustness and execution efficiency. Empirical evaluation across multiple benchmarks demonstrates substantial improvements in tool-call success rates (pass/wins) for both GPT-4 and Claude 3, validating the framework’s effectiveness in improving planning stability and error-recovery efficiency.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Multiagent Planning

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large language models (LLMs) have demonstrated exceptional reasoning capabilities, enabling them to solve various complex problems. Recently, this ability has been applied to the paradigm of tool learning. Tool learning involves providing examples of tool usage and their corresponding functions, allowing LLMs to formulate plans and demonstrate the process of invoking and executing each tool. LLMs can address tasks that they cannot complete independently, thereby enhancing their potential across different tasks. However, this approach faces two key challenges. First, redundant error correction leads to unstable planning and long execution time. Additionally, designing a correct plan among multiple tools is also a challenge in tool learning. To address these issues, we propose Tool-Planner, a task-processing framework based on toolkits. Tool-Planner groups tools based on the API functions with the same function into a toolkit and allows LLMs to implement planning across the various toolkits. When a tool error occurs, the language model can reselect and adjust tools based on the toolkit. Experiments show that our approach demonstrates a high pass and win rate across different datasets and optimizes the planning scheme for tool learning in models such as GPT-4 and Claude 3, showcasing the potential of our method. Our code is public at https://github.com/OceannTwT/Tool-Planner

Problem

• Redundant error correction causes unstable planning.
• Designing correct plans among multiple tools is challenging.
• Tool-Planner groups tools to optimize task planning.

Innovation

• Tool-Planner groups tools into toolkits
• LLMs plan across multiple toolkits
• Error correction via toolkit reselection