MegaAgent: A Large-Scale Autonomous LLM-based Multi-Agent System Without Predefined SOPs

2024-08-19

AI Summary

Existing LLM-driven multi-agent systems (MAS) suffer from weak agent coordination and excessive reliance on manually predefined standard operating procedures (SOPs). To address these limitations, this paper proposes the first SOP-free, fully autonomous MAS architecture, enabling task-complexity-aware dynamic agent generation, automatic task decomposition, parallel execution, and real-time collaborative monitoring. Core technical contributions include LLM-powered agent self-generation, dynamic task graph orchestration, asynchronous message-bus communication, distributed state monitoring, and adaptive load scheduling. Experimental evaluation demonstrates that the system autonomously develops a functional Gomoku game within 800 seconds. In national policy simulation, it scales to 590 agents and generates cross-domain policy proposals with significantly higher quality than MetaGPT, validating its dynamic scalability and cross-domain generalization capability.

Technology Category

• Multiagent Systems: Multiagent Learning
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

LLM-based multi-agent systems (MAS) have shown promise in tackling complex tasks. However, existing solutions often suffer from limited agent coordination and heavy reliance on predefined Standard Operating Procedures (SOPs), which demand extensive human input. To address these limitations, we propose MegaAgent, a large-scale autonomous LLM-based multi-agent system. MegaAgent generates agents based on task complexity and enables dynamic task decomposition, parallel execution, efficient communication, and comprehensive system monitoring of agents. In evaluations, MegaAgent demonstrates exceptional performance, successfully developing a Gobang game within 800 seconds and scaling up to 590 agents in a national policy simulation to generate multi-domain policies. It significantly outperforms existing systems, such as MetaGPT, in both task completion efficiency and scalability. By eliminating the need for predefined SOPs, MegaAgent demonstrates exceptional scalability and autonomy, setting a foundation for advancing true autonomy in MAS. Our code is available at https://github.com/Xtra-Computing/MegaAgent .

Problem

• Enables dynamic task decomposition without predefined SOPs
• Improves agent coordination and communication in multi-agent systems
• Enhances scalability and autonomy for complex task completion

Innovation

• Dynamic task decomposition and parallel execution
• Efficient communication and system monitoring
• Autonomous agent generation without predefined SOPs