From Intent to Execution: Composing Agentic Workflows with Agent Recommendation

2026-05-05

AI Summary

Existing multi-agent systems rely heavily on manual design, resulting in low efficiency and poor scalability. This work proposes an end-to-end automated framework that leverages a large language model (LLM) as a planner to decompose tasks, coupled with a two-stage retrieval-based agent recommendation mechanism—integrating embedding similarity and reranking—to achieve precise task-to-agent matching. To ensure workflow consistency and robustness, the framework incorporates a dynamic invocation graph and a global critical evaluator agent. Experimental results demonstrate that the proposed approach significantly improves task completion recall and outperforms existing methods in both scalability and overall performance, with the critical evaluator further enhancing plan quality.

Abstract

Multi-Agent Systems (MAS) built using AI agents fulfill a variety of user intents that may be used to design and build a family of related applications. However, the creation of such MAS currently involves manual composition of the plan, manual selection of appropriate agents, and manual creation of execution graphs. This paper introduces a framework for the automated creation of multi-agent systems which replaces multiple manual steps with an automated framework. The proposed framework consists of software modules and a workflow to orchestrate the requisite task- specific application. The modules include: an LLM-derived planner, a set of tasks described in natural language, a dynamic call graph, an orchestrator for map agents to tasks, and an agent recommender that finds the most suitable agent(s) from local and global agent registries. The agent recommender uses a two-stage information retrieval (IR) system comprising a fast retriever and an LLM-based re-ranker. We implemented a series of experiments exploring the choice of embedders, re- rankers, agent description enrichment, and supervising critique agent. We benchmarked this system end-to-end, evaluating the combination of planning, agent selection, and task completion, with our proposed approach. Our experimental results show that our approach outperforms the state-of-the- art in terms of the recall rate and is more robust and scalable compared to previous approaches. The critique agent holistically reevaluates both agent and tool recommendations against the overall plan. We show that the inclusion of the critique agent further enhances the recall score, proving that the comprehensive review and revision of task-based agent selection is an essential step in building end-to-end multi-agent systems.

Problem

• Multi-Agent Systems
• Agent Recommendation
• Automated Composition
• Task Planning
• Execution Graph

Innovation

• multi-agent systems
• agent recommendation
• LLM-based planning
• critique agent
• automated orchestration