Search papers, labs, and topics across Lattice.
The paper introduces RankEvolve, a program evolution framework leveraging a large language model to automatically discover improved lexical retrieval algorithms. RankEvolve represents ranking algorithms as executable code, iteratively mutating and recombining them based on retrieval performance on a subset of IR datasets. The evolved algorithms demonstrate novelty, effectiveness, and transferability across various benchmarks, suggesting the viability of LLM-driven program evolution for ranking algorithm discovery.
LLMs can now automatically discover novel and effective ranking algorithms that outperform traditional methods like BM25, suggesting a new paradigm for IR research.
Retrieval algorithms like BM25 and query likelihood with Dirichlet smoothing remain strong and efficient first-stage rankers, yet improvements have mostly relied on parameter tuning and human intuition. We investigate whether a large language model, guided by an evaluator and evolutionary search, can automatically discover improved lexical retrieval algorithms. We introduce RankEvolve, a program evolution setup based on AlphaEvolve, in which candidate ranking algorithms are represented as executable code and iteratively mutated, recombined, and selected based on retrieval performance across 12 IR datasets from BEIR and BRIGHT. RankEvolve starts from two seed programs: BM25 and query likelihood with Dirichlet smoothing. The evolved algorithms are novel, effective, and show promising transfer to the full BEIR and BRIGHT benchmarks as well as TREC DL 19 and 20. Our results suggest that evaluator-guided LLM program evolution is a practical path towards automatic discovery of novel ranking algorithms.
