Search papers, labs, and topics across Lattice.
The paper introduces FAPO (Fully Autonomous Prompt Optimization), a framework designed to optimize multi-step LLM pipelines by diagnosing failures and proposing scoped changes based on intermediate evaluations. By integrating prompt edits with structural modifications when necessary, FAPO effectively addresses bottlenecks that traditional prompt-only optimization overlooks. The results show that FAPO outperforms the baseline GEPA in 15 out of 18 comparisons across various benchmarks, achieving significant accuracy gains, particularly in security-related tasks.
FAPO achieves a remarkable +33.8 percentage point gain in performance by seamlessly transitioning from prompt optimization to structural adjustments in LLM pipelines.
Multi-step LLM pipelines fail through interactions among retrieval, reasoning, and formatting steps, so prompt-only optimization can miss bottlenecks in the chain. We present FAPO (Fully Autonomous Prompt Optimization), a framework that lets Claude Code optimize an LLM pipeline inside a standardized codebase. FAPO evaluates a pipeline, inspects intermediate steps, diagnoses failures, proposes scoped changes, and validates variants repeatedly to optimize against a score function. It first tries prompt edits and, only when prompt optimization appears insufficient, changes chain structure within the permitted scope when attribution identifies a structural bottleneck. Across six benchmarks and three task models, FAPO beats the baseline GEPA in 15 of 18 model-benchmark comparisons. In 11 model-benchmark comparisons, FAPO wins with non-overlapping mean pm trial-standard-deviation ranges, and the mean FAPO-GEPA gain is +14.1 pp. In the six HoVer and IFBench comparisons where prompt-first search escalated to structural changes, FAPO wins all six with a mean gain of +33.8 pp. FAPO also improves performance on security tasks: on CTIBench-RCM, a security CVE-to-CWE task, prompt-only FAPO lifts test accuracy by +4.0 pp on GPT-5, +7.1 pp on Foundation-Sec-8B-Instruct, and +2.0 pp on Foundation-Sec-8B-Reasoning. These results position FAPO as a state-of-the-art pipeline optimization technique for both general-purpose and security-focused tasks.
Stanford HAI