Search papers, labs, and topics across Lattice.
This paper introduces APEX, a framework for automated prompt engineering that optimizes both prompt formulation and data selection to enhance the performance of large language models (LLMs). By dynamically stratifying datasets into Easy, Hard, and Mixed tiers, APEX focuses on the Mixed tier to identify high-leverage data subsets that improve prompt efficacy. The results show that APEX achieves an average performance increase of 11.2% and 6.8% on two different LLMs, underscoring the importance of data efficiency in prompt optimization.
APEX reveals that optimizing data alongside prompts can boost LLM performance by over 11% while significantly reducing wasted compute resources.
Large Language Models are highly sensitive to prompt formulation, necessitating automatic prompt optimization to unlock their full potential. While evolutionary algorithms have emerged as the dominant paradigm, they suffer from a critical bottleneck: data efficiency. Current methods treat the development dataset as a static benchmark, wasting significant compute budget on uninformative data. In this work, we introduce APEX (Automatic Prompt Engineering eXpert), a novel framework that optimizes the data usage alongside the prompt search. APEX dynamically stratifies the dataset into Easy, Hard, and Mixed tiers based on the optimization lineage. By prioritizing the Mixed tier, which identifies the data where the LLM has mixed performance, we identify two high-leverage subsets: the addressable frontier for generating informative mutations and the rank-sensitive frontier for distinguishing candidate quality. We evaluate APEX across three diverse benchmarks: IFBench, SimpleQA Verified, and FACTS Grounding. Under a fixed budget of 5,000 evaluation calls, due to its data efficiency, APEX outperforms the initial prompt by an average of 11.2% on Gemini 2.5 Flash and 6.8% on Gemma 3 27B, demonstrating that a data-centric approach is key to efficient and effective prompt optimization.
Google Research