AgentsNet: Coordination and Collaborative Reasoning in Multi-Agent LLMs

2025-07-11

AI Summary

Existing multi-agent LLM systems lack scalable benchmarks to evaluate how agents self-organize, coordinate reasoning, and communicate efficiently over complex network topologies. Current evaluations are limited to 2–5 agents and ignore structural constraints from distributed systems theory and graph theory. Method: We introduce AgentsNet—the first large-scale multi-agent reasoning benchmark for networks of 2–100 homogeneous LLM agents. It integrates distributed systems protocols (e.g., consensus, leader election) and classical graph problems (e.g., shortest path, connectivity) into a unified evaluation framework, where agent communication is explicitly governed by configurable graph-structured topologies and negotiation mechanisms. Contribution/Results: AgentsNet enables systematic, topology-aware assessment of collaborative reasoning at unprecedented scale. Experiments reveal that state-of-the-art LLMs exhibit robust performance in small networks but suffer substantial degradation as agent count increases—demonstrating scalability bottlenecks. This work establishes the first benchmark for large-scale multi-agent collaboration, offering new empirical insights and analytical tools for advancing decentralized, topology-driven LLM coordination.

Technology Category

• Multiagent Systems: Agent Communication
• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large-language models (LLMs) have demonstrated powerful problem-solving capabilities, in particular when organized in multi-agent systems. However, the advent of such systems also raises several questions on the ability of a complex network of agents to effectively self-organize and collaborate. While measuring performance on standard reasoning benchmarks indicates how well multi-agent systems can solve reasoning tasks, it is unclear whether these systems are able to leverage their topology effectively. Here, we propose AgentsNet, a new benchmark for multi-agent reasoning. By drawing inspiration from classical problems in distributed systems and graph theory, AgentsNet measures the ability of multi-agent systems to collaboratively form strategies for problem-solving, self-organization, and effective communication given a network topology. We evaluate a variety of baseline methods on AgentsNet including homogeneous networks of agents which first have to agree on basic protocols for organization and communication. We find that some frontier LLMs are already demonstrating strong performance for small networks but begin to fall off once the size of the network scales. While existing multi-agent benchmarks cover at most 2-5 agents, AgentsNet is practically unlimited in size and can scale with new generations of LLMs. As such, we also probe frontier models in a setup with up to 100 agents.

Problem

• Assessing multi-agent systems' self-organization and collaboration abilities
• Evaluating effectiveness of network topology in problem-solving
• Scaling multi-agent reasoning benchmarks for large networks

Innovation

• AgentsNet benchmark for multi-agent reasoning
• Measures self-organization and collaborative strategies
• Scales with up to 100 LLM agents