Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces LEVER, a framework for inference-time policy reuse in reinforcement learning, which constructs new policies from a library of pre-trained policies without further environment interaction. LEVER uses behavioral embeddings to retrieve relevant policies and composes them via offline Q-value composition, specifically focusing on scenarios with limited support where value propagation is not possible. Experiments in GridWorld environments demonstrate that LEVER can match or exceed the performance of training from scratch, while also identifying limitations in handling long-horizon dependencies due to the lack of value propagation.
Forget retraining: LEVER lets you snap together pre-trained RL policies at inference time, matching or beating from-scratch performance in some cases.
Reinforcement learning (RL) policies are typically trained for fixed objectives, making reuse difficult when task requirements change. We study inference-time policy reuse: given a library of pre-trained policies and a new composite objective, can a high-quality policy be constructed entirely offline, without additional environment interaction? We introduce lever (Leveraging Efficient Vector Embeddings for Reusable policies), an end-to-end framework that retrieves relevant policies, evaluates them using behavioral embeddings, and composes new policies via offline Q-value composition. We focus on the support-limited regime, where no value propagation is possible, and show that the effectiveness of reuse depends critically on the coverage of available transitions. To balance performance and computational cost, lever proposes composition strategies that control the exploration of candidate policies. Experiments in deterministic GridWorld environments show that inference-time composition can match, and in some cases exceed, training-from-scratch performance while providing substantial speedups. At the same time, performance degrades when long-horizon dependencies require value propagation, highlighting a fundamental limitation of offline reuse.