Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces Prioritized Optimization with Local Contextual Aggregation (POLCA), a framework that uses a generative language model as an optimizer to tackle stochastic generative optimization problems like LLM prompt engineering and multi-turn agent design. POLCA employs a priority queue for managing exploration-exploitation, an $\varepsilon$-Net mechanism for parameter diversity, and an LLM summarizer for meta-learning across trials, which allows it to handle noisy feedback and stochastic system behaviors. Empirical results across benchmarks like $\tau$-bench and HotpotQA demonstrate POLCA's superior sample efficiency and performance compared to existing state-of-the-art algorithms in both deterministic and stochastic settings.
Forget manual iteration: POLCA uses LLMs to automatically optimize complex systems, achieving state-of-the-art results in tasks ranging from prompt engineering to CUDA kernel generation.
Optimizing complex systems, ranging from LLM prompts to multi-turn agents, traditionally requires labor-intensive manual iteration. We formalize this challenge as a stochastic generative optimization problem where a generative language model acts as the optimizer, guided by numerical rewards and text feedback to discover the best system. We introduce Prioritized Optimization with Local Contextual Aggregation (POLCA), a scalable framework designed to handle stochasticity in optimization -- such as noisy feedback, sampling minibatches, and stochastic system behaviors -- while effectively managing the unconstrained expansion of solution space. POLCA maintains a priority queue to manage the exploration-exploitation tradeoff, systematically tracking candidate solutions and their evaluation histories. To enhance efficiency, we integrate an $\varepsilon$-Net mechanism to maintain parameter diversity and an LLM Summarizer to perform meta-learning across historical trials. We theoretically prove that POLCA converges to near-optimal candidate solutions under stochasticity. We evaluate our framework on diverse benchmarks, including $\tau$-bench, HotpotQA (agent optimization), VeriBench (code translation) and KernelBench (CUDA kernel generation). Experimental results demonstrate that POLCA achieves robust, sample and time-efficient performance, consistently outperforming state-of-the-art algorithms in both deterministic and stochastic problems. The codebase for this work is publicly available at https://github.com/rlx-lab/POLCA.