FlowBot: Inducing LLM Workflows with Bilevel Optimization and Textual Gradients

2026-04-28

AI Summary

This work proposes a data-driven, end-to-end approach to automatically construct and optimize large language model (LLM) workflows, addressing the deployment bottlenecks associated with manual pipeline design. The workflow construction is formulated as a bilevel optimization problem: the outer loop searches over high-level structural configurations, while the inner loop performs differentiable optimization of individual LLM invocation modules using textual gradients, enabling layer-wise adjustments analogous to backpropagation. This is the first method to integrate bilevel optimization with textual gradients, allowing efficient workflows to be discovered fully automatically without human intervention. Experimental results demonstrate that the proposed approach achieves performance on par with or superior to strong baseline systems that rely on either handcrafted or automatically generated workflows across multiple tasks.

Abstract

LLM workflows, which coordinate structured calls to individual LLMs (each augmented with varying instructions and tools) to achieve a particular goal, offer a promising path towards extending the capabilities of LLMs and building powerful systems that can tackle diverse tasks. However, existing approaches for building such workflows generally rely on human-crafted pipelines and prompts, which presents a substantial bottleneck in real world deployment. How can automatically induce and optimize such workflows in a data-driven way? This paper describes a simple data-driven approach for automatically inducing LLM workflows. We formulate workflow induction as a bilevel optimization problem: an outer loop which optimizes a high-level sketch of the workflow (in particular how the LLM calls should be structured), and an inner loop which optimizes each individual LLM call one-by one. Both loops are optimized with ``textual gradients'' where for the inner loop we optimize each component in a modular way through ``backpropagating'' textual gradients layer-by-layer. We find that LLM workflows discovered through our \textsc{FlowBot} (work\textbf{flow} induction through \textbf{b}ilevel \textbf{o}ptimization and \textbf{t}extual gradients) approach performs competitively against strong baselines that make use of human-crafted or automatically-generated workflows.

Problem

• LLM workflows
• workflow induction
• bilevel optimization
• textual gradients
• automated pipeline

Innovation

• bilevel optimization
• textual gradients
• LLM workflows
• automated workflow induction
• modular optimization