Search papers, labs, and topics across Lattice.
Code-Space Response Oracles (CSRO) replaces deep reinforcement learning oracles in Policy-Space Response Oracles (PSRO) with Large Language Models (LLMs) to generate interpretable multi-agent policies. CSRO reframes the best response computation as a code generation task, prompting an LLM to produce policies directly as human-readable code. Experiments demonstrate that CSRO achieves competitive performance while producing a diverse set of explainable policies, using techniques like zero-shot prompting, iterative refinement, and a distributed LLM-based evolutionary system called AlphaEvolve.
Forget black-box policies: CSRO uses LLMs to generate human-readable code policies in multi-agent RL, achieving performance competitive with traditional methods.
Recent advances in multi-agent reinforcement learning, particularly Policy-Space Response Oracles (PSRO), have enabled the computation of approximate game-theoretic equilibria in increasingly complex domains. However, these methods rely on deep reinforcement learning oracles that produce `black-box'neural network policies, making them difficult to interpret, trust or debug. We introduce Code-Space Response Oracles (CSRO), a novel framework that addresses this challenge by replacing RL oracles with Large Language Models (LLMs). CSRO reframes the best response computation as a code generation task, prompting an LLM to generate policies directly as human-readable code. This approach not only yields inherently interpretable policies but also leverages the LLM's pretrained knowledge to discover complex, human-like strategies. We explore multiple ways to construct and enhance an LLM-based oracle: zero-shot prompting, iterative refinement and \emph{AlphaEvolve}, a distributed LLM-based evolutionary system. We demonstrate that CSRO achieves performance competitive with baselines while producing a diverse set of explainable policies. Our work presents a new perspective on multi-agent learning, shifting the focus from optimizing opaque policy parameters to synthesizing interpretable algorithmic behavior.
DeepMind