AFlow: Automating Agentic Workflow Generation

2024-10-14

arXiv.org

AI Summary

Existing LLM-based agent workflows rely on manually designed instructions and action sequences, suffering from poor scalability. This paper proposes the first end-to-end fully automated workflow generation framework, formulating workflow optimization as a tree search problem in code representation space. Our method innovatively integrates Monte Carlo Tree Search (MCTS), LLM-driven node orchestration, and execution-feedback-guided code-level iterative refinement—requiring no human initialization. It achieves complete automation from task input to executable workflow generation and optimization. Evaluated on six benchmark tasks, our approach improves average accuracy by 5.7% over prior methods. Notably, a lightweight model achieves performance superior to GPT-4o while incurring only 4.55% of its inference cost.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Planning, Routing, and Scheduling: Planning with Language Models
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Large language models (LLMs) have demonstrated remarkable potential in solving complex tasks across diverse domains, typically by employing agentic workflows that follow detailed instructions and operational sequences. However, constructing these workflows requires significant human effort, limiting scalability and generalizability. Recent research has sought to automate the generation and optimization of these workflows, but existing methods still rely on initial manual setup and fall short of achieving fully automated and effective workflow generation. To address this challenge, we reformulate workflow optimization as a search problem over code-represented workflows, where LLM-invoking nodes are connected by edges. We introduce AFlow, an automated framework that efficiently explores this space using Monte Carlo Tree Search, iteratively refining workflows through code modification, tree-structured experience, and execution feedback. Empirical evaluations across six benchmark datasets demonstrate AFlow's efficacy, yielding a 5.7% average improvement over state-of-the-art baselines. Furthermore, AFlow enables smaller models to outperform GPT-4o on specific tasks at 4.55% of its inference cost in dollars. The code will be available at https://github.com/geekan/MetaGPT.

Problem

• Automate agentic workflow generation
• Optimize workflows using Monte Carlo Tree Search
• Reduce human effort and improve scalability

Innovation

• Monte Carlo Tree Search
• code-represented workflows
• iterative refinement