Search papers, labs, and topics across Lattice.
This paper introduces ESPO, an innovative approach to prompt optimization that addresses the issue of prompt bloat seen in existing methods like GEPA by decomposing the process into three structured phases: Diagnose, Propose, and Select. By effectively clustering training errors, generating diverse candidate prompts, and applying bootstrap stability selection, ESPO achieves a significant accuracy improvement of +3.76 percentage points over the state-of-the-art while producing prompts that are 47% shorter. The method consistently outperforms GEPA across multiple NLP benchmarks and models, demonstrating its robustness and efficiency in prompt optimization.
ESPO not only boosts accuracy by nearly 4 percentage points but also slashes prompt length by nearly half, redefining efficiency in prompt optimization.
Evolutionary prompt optimizers such as GEPA suffer from prompt bloat: each iteration appends rules and caveats, producing prompts up to 3$\times$ longer yet no more accurate. We trace this to three deficiencies - incomplete error observation, limited search diversity, and unreliable selection - and propose ESPO (Error-Structured Prompt Optimization), which decomposes prompt optimization into three phases: Diagnose clusters all training errors into structural patterns in one round; Propose generates candidates via four complementary strategies with independent biases; Select applies bootstrap stability selection. On seven public NLP benchmarks - Tweet, MMLU, GSM8K, HotpotQA, ScoNe, HoVer, and PUPA - ESPO improves average accuracy by $+$3.76 pp over the state-of-the-art (74.67% vs 70.91% for GEPA), matching or exceeding GEPA on every dataset while producing prompts 47% shorter (1,004 vs 1,878 chars) and faster at inference. Cross-model experiments across four additional student models (Gemma 3 12B, Mistral 14B, Qwen3 32B, Claude Haiku 4.5) show ESPO yields the best average accuracy on every model tested, with the largest gap on Qwen3 GSM8K (15.00% $\to$ 91.40%). A generalization bound (Appendix) grounds each phase in a corresponding term of the test-time gap, and the ablation confirms a key prediction: adding diversity without bootstrap selection actually hurts performance ($-$1.20%).
