Search papers, labs, and topics across Lattice.
This paper introduces the PACT architecture, which integrates a reactive reinforcement learning (RL) policy with a deliberative Small Language Model (SLM) planner to enhance decision-making in unfamiliar environments. By asynchronously generating and validating action plans through the SLM, PACT ensures that only safe and feasible plans are executed, effectively bypassing the need for RL policy retraining. The approach demonstrates superior performance across various FrozenLake configurations, highlighting the effectiveness of combining deliberative planning with reactive execution.
A hybrid architecture that leverages deliberative planning can significantly outperform traditional reactive RL policies in complex environments.
Reinforcement Learning (RL) policies often degrade in unfamiliar environments because they lack explicit deliberation. We propose Plan, Align, Commit, Think (PACT), a hybrid architecture that combines a fast, reactive RL policy with a slow, deliberative Small Language Model (SLM) planner. PACT invokes the SLM asynchronously to generate and validate candidate action plans. Once a plan is verified through simulation as safe, feasible, and complete, it is executed directly, bypassing the RL policy without retraining or modifying it. Evaluated on three FrozenLake configurations of increasing difficulty, PACT outperforms all baselines while relying on a 2B-parameter SLM backbone, suggesting that deliberative planning and reactive execution are more powerful in concert than either is alone in these settings.
