The Illusion of Agentic Complexity in README.md Generation: Evaluating Single-Agent vs. Multi-Agent RAG Systems

2026-06-29

AI Summary

This study investigates whether multi-agent systems outperform single-agent approaches in the task of automatically generating README files for GitHub repositories, while examining the trade-offs between architectural complexity and performance in terms of efficiency and output quality. Building upon a retrieval-augmented generation (RAG) framework, the authors systematically compare three architectures: single-agent, multi-agent, and a developer-guided planning approach (DevPlan), with LARCH as the baseline. Experimental results demonstrate that the multi-agent system achieves 98% structural consistency but incurs significantly higher computational costs; in contrast, the single-agent variant matches its text quality while reducing token consumption by 86% and doubling generation speed. Notably, the lightweight DevPlan configuration delivers the best overall performance, substantially surpassing other setups and providing the first empirical evidence that increased architectural complexity is not always necessary for superior outcomes.

Abstract

Large Language Models (LLMs) are increasingly utilized to automate several software engineering tasks, including code completion, code summarization, testing, and the generation of repository-level documentation. While Multi-Agent Systems (MAS) are often adopted to support such tasks under the premise that task decomposition improves performance, the impact of architectural complexity on practical efficiency remains under-examined. This study empirically evaluates Retrieval-Augmented Generation (RAG) dependent architectures for the generation of README files for GitHub repositories. In this work, we conducted a systematic comparison between a Single-Agent pipeline, a specialized MAS, and a developer-guided planning (DevPlan) variant, benchmarked against LARCH -- a state-of-the-art baseline -- and the original ground truth. Results indicate a critical architectural trade-off: the Single-Agent pipeline achieves lexical quality comparable to MAS while reducing token consumption by 86% and operating at twice the speed. In contrast, manual taxonomy analysis demonstrates that MAS achieves high structural consistency (98%), resolving formatting issues observed in single-agent approaches. Autonomous planning is identified as the primary pipeline bottleneck; incorporating lightweight developer-guided plans produces the highest overall documentation quality, surpassing all the analyzed configurations.

Problem

• README generation
• Multi-Agent Systems
• Retrieval-Augmented Generation
• architectural complexity
• software documentation

Innovation

• Multi-Agent Systems
• Retrieval-Augmented Generation
• README generation
• developer-guided planning
• architectural trade-off