FAPO: Fully Autonomous Prompt Optimization of Multi-Step LLM Pipelines

2026-06-17

AI Summary

This work addresses the structural fragility of multi-step large language model (LLM) pipelines, which often fail at stages such as retrieval, reasoning, or formatting due to inherent bottlenecks that cannot be resolved by prompt engineering alone. The authors propose FAPO, a novel framework that enables fully automated joint optimization of both prompts and pipeline structure for the first time. FAPO employs a failure-attribution mechanism to diagnose errors in intermediate steps, prioritizing prompt editing and resorting to structural adjustments only when necessary, with iterative refinement guided by a scoring function. Evaluated across 18 model–benchmark combinations, FAPO outperforms baselines in 15 cases (average gain: 14.1%) and achieves consistent superiority in all six tasks requiring structural modifications (average gain: 33.8%), while also significantly improving accuracy on the CTIBench-RCM safety benchmark.

Abstract

Multi-step LLM pipelines fail through interactions among retrieval, reasoning, and formatting steps, so prompt-only optimization can miss bottlenecks in the chain. We present FAPO (Fully Autonomous Prompt Optimization), a framework that lets Claude Code optimize an LLM pipeline inside a standardized codebase. FAPO evaluates a pipeline, inspects intermediate steps, diagnoses failures, proposes scoped changes, and validates variants repeatedly to optimize against a score function. It first tries prompt edits and, only when prompt optimization appears insufficient, changes chain structure within the permitted scope when attribution identifies a structural bottleneck. Across six benchmarks and three task models, FAPO beats the baseline GEPA in 15 of 18 model-benchmark comparisons. In 11 model-benchmark comparisons, FAPO wins with non-overlapping mean $\pm$ trial-standard-deviation ranges, and the mean FAPO-GEPA gain is +14.1 pp. In the six HoVer and IFBench comparisons where prompt-first search escalated to structural changes, FAPO wins all six with a mean gain of +33.8 pp. FAPO also improves performance on security tasks: on CTIBench-RCM, a security CVE-to-CWE task, prompt-only FAPO lifts test accuracy by +4.0 pp on GPT-5, +7.1 pp on Foundation-Sec-8B-Instruct, and +2.0 pp on Foundation-Sec-8B-Reasoning. These results position FAPO as a state-of-the-art pipeline optimization technique for both general-purpose and security-focused tasks.

Problem

• multi-step LLM pipelines
• prompt optimization
• structural bottlenecks
• pipeline failure
• autonomous optimization

Innovation

• prompt optimization
• LLM pipeline
• autonomous optimization
• structural bottleneck
• multi-step reasoning