An Evaluation of Role-Based Multi-Agent Code Generation on Repository-Scale Problems

2026-07-05

AI Summary

This work addresses the limitations of large language models (LLMs) in handling warehouse-scale, complex programming tasks by proposing a role-based multi-agent collaboration framework. It presents the first systematic evaluation of such an architecture on real-world Java software repository tasks. The approach assigns distinct roles to multiple LLMs and integrates code similarity analysis with repository-level program understanding to collaboratively generate code that better aligns with developer coding styles. Experimental results demonstrate that this method outperforms single-model baselines in terms of code style consistency. However, it still falls significantly short of human programmers in overall implementation quality, thereby revealing both the potential and current limitations of multi-agent code generation in realistic software development contexts.

Abstract

Role-based multiagent code generation aims to make LLMs more effective on repository-scale problems, moving beyond small programming tasks. We evaluate this approach on 12 Java repositories, finding greater similarity to developer code than single LLMs, but a persistent gap from human implementations.

Problem

• role-based multi-agent
• code generation
• repository-scale problems
• LLMs
• human-like code

Innovation

• role-based multi-agent
• code generation
• repository-scale
• large language models
• software engineering