Smurfs: Leveraging Multiple Proficiency Agents with Context-Efficiency for Tool Planning

2024-05-09

arXiv.org

AI Summary

To address rollback instability, context redundancy, and premature termination in single-agent deep-first search decision trees (DFSDTs), this paper proposes a lightweight, parameter-free multi-agent framework that dynamically decomposes a single large language model into a collaborative cluster of specialized agents. Methodologically, it introduces: (1) Context-Aware Role Prompting (CRP), enabling zero-cost, efficient role assignment; (2) dynamic tool routing and multi-stage collaborative planning; and (3) an interpretable agent division paradigm. Evaluated on StableToolBench (open-ended tasks) and HotpotQA (closed-ended tasks), the framework achieves state-of-the-art performance, significantly outperforming all baselines. Ablation studies confirm that each component contributes over 87% to the overall improvement. The approach enhances both robustness and transparency in LLM-based reasoning without introducing trainable parameters or increasing inference latency substantially.

Technology Category

• Multiagent Systems: Multiagent Planning
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

The emergence of large language models (LLMs) has opened up unprecedented possibilities for automating complex tasks that are often comparable to human performance. Despite their capabilities, LLMs still encounter difficulties in completing tasks that require high levels of accuracy and complexity due to their inherent limitations in handling multifaceted problems single-handedly. This paper introduces `Smurfs', a cutting-edge multi-agent framework designed to revolutionize the application of LLMs. By seamlessly transforming a conventional LLM into a synergistic multi-agent ensemble, Smurfs can enhance the model's ability to solve complex tasks at no additional cost. This is achieved through innovative prompting strategies that allocate distinct roles within the model, thereby facilitating collaboration among specialized agents and forming an intelligent multi-agent system. Our empirical investigation on both open-ended task of StableToolBench and closed-ended task on HotpotQA showcases Smurfs' superior capability in intricate tool utilization scenarios. Notably, Smurfs outmatches all the baseline methods in both experiments, setting new state-of-the-art performance. Furthermore, through comprehensive ablation studies, we dissect the contribution of the core components of the multi-agent framework to its overall efficacy. This not only verifies the effectiveness of the framework, but also sets a route for future exploration of multi-agent LLM systems.

Problem

• Enhance tool use in multi-agent systems for complex problem-solving
• Reduce token usage and improve efficiency in DFSDT framework
• Address rollback instability and premature termination in single-agent DFSDT

Innovation

• Multi-agent system enhances DFSDT efficiency
• Modular design reduces token usage significantly
• Training-free approach improves tool planning performance