Multi-Agent Software Development through Cross-Team Collaboration

2024-06-13

arXiv.org

AI Summary

Existing LLM-based multi-agent systems (e.g., ChatDev) employ a linear, single-path collaboration paradigm, where each stage produces only one output—leading to limited exploration of the solution space, suboptimal outcomes, and insufficient diversity in development trajectories. Method: This paper proposes the Cross-Team Collaboration (CTC) framework, introducing the first paradigm wherein multiple specialized, role-assigned LLM agent teams concurrently execute requirement analysis, design, coding, and testing. CTC enables dynamic exploration of diverse decision paths and cross-team knowledge sharing via asynchronous inter-team communication and a waterfall-enhanced consensus protocol. Contribution/Results: Experiments demonstrate significant improvements over state-of-the-art baselines on software generation tasks; story generation benchmarks confirm strong cross-domain generalization; all code and data are publicly released.

Technology Category

• Multiagent Systems: Teamwork
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Humans and AI: Human-AI Collaboration / Human-AI Teaming

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

The latest breakthroughs in Large Language Models (LLMs), eg., ChatDev, have catalyzed profound transformations, particularly through multi-agent collaboration for software development. LLM agents can collaborate in teams like humans, and follow the waterfall model to sequentially work on requirements analysis, development, review, testing, and other phases to perform autonomous software generation. However, for an agent team, each phase in a single development process yields only one possible outcome. This results in the completion of only one development chain, thereby losing the opportunity to explore multiple potential decision paths within the solution space. Consequently, this may lead to obtaining suboptimal results. To address this challenge, we introduce Cross-Team Collaboration (CTC), a scalable multi-team framework that enables orchestrated teams to jointly propose various decisions and communicate with their insights in a cross-team collaboration environment for superior content generation. Experimental results in software development reveal a notable increase in quality compared to state-of-the-art baselines, underscoring the efficacy of our framework. The significant improvements in story generation demonstrate the promising generalization ability of our framework across various domains. We anticipate that our work will guide LLM agents towards a cross-team paradigm and contribute to their significant growth in but not limited to software development. The code and data will be available at https://github.com/OpenBMB/ChatDev.

Problem

• Limited outcomes from single-agent teams in software development
• Lack of exploration for multiple decision paths in solutions
• Need for scalable multi-team collaboration to generate superior solutions

Innovation

• Multi-team framework for diverse solution proposals
• Cross-team collaboration with self-independence interaction
• Scalable orchestration for superior solution generation