AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answering

2025-10-06

AI Summary

Existing multi-agent question-answering systems suffer from task-agnostic agent configuration selection and neglect of fine-grained semantic structures, leading to routing uncertainty. Method: This paper proposes a knowledge graph-guided dynamic routing framework. It models each QA instance as a heterogeneous knowledge graph, employs a heterogeneous graph neural network for structured cross-node semantic propagation, and jointly optimizes the routing distribution via soft supervision and weighted output aggregation guided by empirical performance signals. Contribution/Results: The framework significantly outperforms both single-agent and ensemble baselines across multiple benchmark datasets and diverse large language model backbones. Empirical results demonstrate its strong task adaptivity, explicit structural awareness, and robust generalization capability.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Natural Language Processing: Question Answering
• Multiagent Systems: Adversarial Agents

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Large language models (LLMs) and agent-based frameworks have advanced rapidly, enabling diverse applications. Yet, with the proliferation of models and agentic strategies, practitioners face substantial uncertainty in selecting the best configuration for a downstream task. Prior studies show that different agents and backbones exhibit complementary strengths, and that larger models are not always superior, underscoring the need for adaptive routing mechanisms. Existing approaches to agent routing, however, often emphasize cost efficiency while overlooking the fine-grained contextual and relational structure inherent in QA tasks. In this paper, we propose tAgentRouter, a framework that formulates multi-agent QA as a knowledge-graph-guided routing problem supervised by empirical performance signals. Specifically, we convert QA instance into a knowledge graph that jointly encodes queries, contextual entities, and agents, and then train a heterogeneous graph neural network (GNN) to propagate information across node types and produce task-aware routing distributions over agents. By leveraging soft supervision and weighted aggregation of agent outputs, AgentRouter learns principled collaboration schemes that capture the complementary strengths of diverse agents. Extensive experiments demonstrate that our framework consistently outperforms single-agent and ensemble baselines, while generalizing across benchmarks and LLM backbones. These results highlight the effectiveness and robustness of graph-supervised multi-agent routing for question answering.

Problem

• Selecting optimal agent configurations for question answering tasks
• Overlooking contextual and relational structure in agent routing
• Leveraging complementary strengths of diverse agents in collaboration

Innovation

• Knowledge graph guides multi-agent routing
• Heterogeneous GNN propagates cross-node information
• Soft supervision aggregates complementary agent outputs