Search papers, labs, and topics across Lattice.
This paper analyzes LLM-guided evolutionary search across 15 LLMs and 8 tasks, finding that strong optimizers act as local refiners, making frequent incremental improvements while maintaining semantic coherence. Weaker optimizers, in contrast, exhibit significant semantic drift and stagnation despite occasional breakthroughs. The study reveals that solution novelty alone does not predict performance; rather, it is the combination of novelty and localized search that drives successful optimization.
LLMs that excel at evolutionary search aren't just smart zero-shot reasoners; they're masters of "local refinement," making small, consistent improvements without wandering off into the semantic wilderness.
Recent work has demonstrated the promise of orchestrating large language models (LLMs) within evolutionary and agentic optimization systems. However, the mechanisms driving these optimization gains remain poorly understood. In this work, we present a large-scale study of LLM-guided evolutionary search, collecting optimization trajectories for 15 LLMs across 8 tasks. Although zero-shot problem-solving ability correlates with final optimization outcomes, it explains only part of the variance: models with similar initial capability often induce dramatically different search trajectories and outcomes. By analyzing these trajectories, we find that strong LLM optimizers behave as local refiners, producing frequent incremental improvements while progressively localizing the search in semantic space. Conversely, weaker optimizers exhibit large semantic drift, with sporadic breakthroughs followed by stagnation. Notably, various measures of solution novelty do not predict final performance; novelty is beneficial only when the search remains sufficiently localized around high-performing regions of the solution space. Our results highlight the importance of trajectory analysis for understanding and improving LLM-based optimization systems and provide actionable insights for their design and training.
