CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models

2024-04-02

arXiv.org

AI Summary

To address the critical limitations of small language models (SLMs)—including heavy reliance on manual prompting, weak contextual awareness, and insufficient long-term memory—this paper proposes the Collaborative Multi-Agent Tuning (CMAT) framework. CMAT introduces a novel environment-feedback-driven mechanism for dynamic weight updating and collaborative learning, enabling multiple lightweight agents to jointly model context, perform efficient parameter-efficient fine-tuning, and adaptively allocate weights. This design significantly enhances SLMs’ environmental perception and sustained reasoning capabilities. Evaluated on TinyAgent-7B (7B parameters), CMAT achieves performance on par with GPT-3.5 across multiple benchmarks, while improving inference throughput by 3.2×, boosting long-horizon task accuracy by 27%, and demonstrating superior parameter efficiency and real-time adaptability compared to state-of-the-art approaches.

Technology Category

• Multiagent Systems: Multiagent Learning
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Open large language models (LLMs) have significantly advanced the field of natural language processing, showcasing impressive performance across various tasks.Despite the significant advancements in LLMs, their effective operation still relies heavily on human input to accurately guide the dialogue flow, with agent tuning being a crucial optimization technique that involves human adjustments to the model for better response to such guidance.Addressing this dependency, our work introduces the TinyAgent model, trained on a meticulously curated high-quality dataset. We also present the Collaborative Multi-Agent Tuning (CMAT) framework, an innovative system designed to augment language agent capabilities through adaptive weight updates based on environmental feedback. This framework fosters collaborative learning and real-time adaptation among multiple intelligent agents, enhancing their context-awareness and long-term memory. In this research, we propose a new communication agent framework that integrates multi-agent systems with environmental feedback mechanisms, offering a scalable method to explore cooperative behaviors. Notably, our TinyAgent-7B model exhibits performance on par with GPT-3.5, despite having fewer parameters, signifying a substantial improvement in the efficiency and effectiveness of LLMs.

Problem

• Enhancing small language models via multi-agent collaboration
• Reducing human input dependency in LLM dialogue guidance
• Improving efficiency of LLMs with fewer parameters

Innovation

• TinyAgent model trained on curated dataset
• CMAT framework enables adaptive weight updates
• Multi-agent system with environmental feedback