Search papers, labs, and topics across Lattice.
This paper introduces an evolutionary framework, REvolve, that leverages large language models to discover novel reinforcement learning algorithms by directly searching over executable update rules. The search intentionally excludes canonical RL mechanisms to encourage the discovery of non-standard learning rules, which are then refined via LLM-proposed hyperparameter ranges. Evaluated on Gymnasium benchmarks, the discovered algorithms achieve competitive performance compared to established baselines like SAC and PPO, demonstrating the potential of LLMs to automate RL algorithm design.
Forget hand-designed RL algorithms – LLMs can evolve competitive learners from scratch, even when forced to invent completely new update rules.
Reinforcement learning algorithms are defined by their learning update rules, which are typically hand-designed and fixed. We present an evolutionary framework for discovering reinforcement learning algorithms by searching directly over executable update rules that implement complete training procedures. The approach builds on REvolve, an evolutionary system that uses large language models as generative variation operators, and extends it from reward-function discovery to algorithm discovery. To promote the emergence of nonstandard learning rules, the search excludes canonical mechanisms such as actor--critic structures, temporal-difference losses, and value bootstrapping. Because reinforcement learning algorithms are highly sensitive to internal scalar parameters, we introduce a post-evolution refinement stage in which a large language model proposes feasible hyperparameter ranges for each evolved update rule. Evaluated end-to-end by full training runs on multiple Gymnasium benchmarks, the discovered algorithms achieve competitive performance relative to established baselines, including SAC, PPO, DQN, and A2C.
