Graph-of-Agents: A Graph-based Framework for Multi-Agent LLM Collaboration

2026-04-18

AI Summary

Existing approaches to multi-agent large language model collaboration exhibit limitations in agent selection, communication efficiency, and response fusion. This work proposes the first graph-based framework for modeling multi-agent collaboration, dynamically selecting relevant agents via model cards and constructing a directed graph grounded in response correlations. It introduces a bidirectional message-passing mechanism coupled with graph pooling strategies—such as max or mean pooling—to enable efficient and effective response integration. The proposed method achieves substantial performance gains across multiple benchmarks, including MMLU, MMLU-Pro, GPQA, MATH, HumanEval, and MedMCQA, surpassing current state-of-the-art methods that utilize all six available agents by employing only three carefully selected agents.

Technology Category

• Multiagent Systems: Agent Communication
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Agentic search

Abstract

With an ever-growing zoo of LLMs and benchmarks, the need to orchestrate multiple models for improved task performance has never been more pressing. While frameworks like Mixture-of-Agents (MoA) attempt to coordinate LLMs, they often fall short in terms of (1) selecting relevant agents, (2) facilitating effective intra-agent communication, and (3) integrating responses efficiently. In this work, we propose Graph-of-Agents (GoA), a new graph-based framework for modeling multi-agent LLM communication. Our approach begins with node sampling, selecting only the most relevant agents by leveraging model cards that summarize each model's domain, task specialization, and other characteristics. Next, we construct edges between the selected agents by evaluating their responses against one another to determine relevance ordering. Directed message passing is then performed from highly relevant agents to less relevant ones to enhance their responses, followed by reverse message passing to refine the original responses of the more relevant agents. Finally, the updated responses are aggregated via graph-based pooling (e.g., max or mean pooling) to produce a single, unified answer. We evaluate GoA on diverse multi-domain benchmarks (MMLU, MMLU-Pro, GPQA) and domain-specific benchmarks (MATH, HumanEval, MedMCQA), with an agent pool of 6 LLMs spanning multiple domains. Surprisingly, GoA achieves superior performance using only 3 selected agents, outperforming recent multi-agent LLM baselines that utilize all 6 agents simultaneously. By adopting a graph structure, GoA offers both scalability and effectiveness through structured message passing-positioning it as a strong candidate for navigating the challenges of the ever-growing LLM zoo. Code is available at: https://github.com/UNITES-Lab/GoA.

Problem

• Multi-Agent LLM Collaboration
• Agent Selection
• Intra-Agent Communication
• Response Integration
• LLM Orchestration

Innovation

• Graph-of-Agents
• multi-agent LLM collaboration
• graph-based message passing
• agent selection
• response aggregation