SwarmBench: Can Large Language Models Act as Agent Swarm Orchestrators?

2026-08-31

AI Summary

研究提出SwarmBench评估大型语言模型在动态编排Agent群中的表现，并通过SwarmExp方法提升其编排性能。

Abstract

Large language model-based multi-agent systems are evolving from fixed interaction topologies toward dynamically orchestrated Agent Swarms. However, existing benchmarks are still largely based on single-agent or general-purpose agent tasks, making it difficult to systematically evaluate key orchestration capabilities. We propose SwarmBench, a benchmark that evaluates model performance from multiple perspectives, including accuracy, efficiency, cost, and process quality. Experimental results show that current models exhibit substantial differences in orchestration capability. These differences are reflected not only in final accuracy, efficiency, and cost, but also in the overall quality of the orchestration process itself. Based on these findings, we further propose SwarmExp, a simple yet effective method based on experience extraction and experience replay, which consistently improves the orchestration performance of large language models.

Problem

• Large language models
• Agent Swarms
• Orchestration capabilities
• Benchmark

Innovation

• SwarmBench
• Agent Swarms
• Orchestration
• SwarmExp
• Experience Replay