Search papers, labs, and topics across Lattice.
This paper introduces EpicStar, a novel framework that enhances strategic reasoning in Large Language Models (LLMs) by integrating memory as a policy for long-horizon decision-making. By maintaining a bank of successful past episodes and utilizing a dynamic gating mechanism, EpicStar allows agents to effectively balance between executing retrieved actions and performing new reasoning based on current environmental changes. Evaluated in StarCraft II, EpicStar significantly outperforms baseline methods, achieving higher win rates with substantially reduced token consumption across various opponent strategies and difficulty levels.
Memory-enhanced agency enables LLMs to achieve robust long-term strategic execution, outperforming traditional methods in dynamic environments.
Strategic reasoning in Large Language Models (LLMs) within long-horizon environments is often limited by inconsistent subgoals. In these settings, finite attention resources prevent the model from maintaining strategic coherence over thousands of steps. This limitation leads to strategic drift, where localized decisions fail to sustain a coherent trajectory across reasoning. To address this, we introduce EpicStar, a framework that enables agents to learn memory as policy to tackle long-horizon reasoning. Specifically, the agent maintains a bank of successful past episodes as a heuristic alongside a working memory to track short-term environmental changes. During inference, a dynamic gating mechanism determines whether to execute a retrieved action directly or to perform new reasoning through a contextual fusion of the retrieved episodes and current working memory. Utilizing StarCraft II as the testbed, we evaluated EpicStar against diverse opponent styles. It significantly outperforms baseline methods, achieving higher win rates while consuming an order of magnitude fewer tokens, and it maintains this advantage consistently across difficulty levels and opponent strategies. Our findings provide compelling evidence that structured cross-episode memory is essential for enabling LLM agents to perform robust, long-term strategic execution in dynamic, autonomous settings.
