Search papers, labs, and topics across Lattice.
This paper introduces Stochastic Prompt Optimization (SPO), a novel framework for automatic prompt optimization that treats the prompt space as a black box for stochastic search. The authors evaluate three strategies, including SAGE, a multi-agent approach that integrates diagnostic code execution, and find that the effectiveness of each strategy is contingent on the landscape structure and error type. Notably, SAGE is successfully applied to optimize a mental-health chatbot, resulting in a statistically significant improvement in user retention through a continuous optimization process involving multiple A/B tests.
SAGE achieves a statistically robust increase in user retention for a mental-health chatbot by effectively navigating the complexities of prompt optimization through agent-guided exploration.
Context engineering has emerged as a primary lever for improving AI systems without parameter updates. Recent work showing that textual gradients do not function as real gradients motivates treating automatic prompt optimization (APO) as black-box search. We introduce SPO (Stochastic Prompt Optimization), a framework for stochastic search over prompt space, and compare three strategies of increasing sophistication: error-informed random search, a genetic algorithm with evolutionary operators, and SAGE (SPO via Agent-Guided Exploration), a multi-agent pipeline with diagnostic code execution. Across three benchmarks, no single strategy dominates; effectiveness depends on the interaction of landscape structure with error type. We further deploy SAGE on a mental-health chatbot under a continuous optimization paradigm, where it compounds eight cycles of individually-noisy A/B tests into a statistically robust gain in next-day retention. We argue that coupling qualitative diagnosis with quantitative validation is what makes agentic optimization effective for open-ended task-oriented dialogue.
