Search papers, labs, and topics across Lattice.
This paper introduces a novel LLM agent framework that enhances planning in interactive environments by using in-context learning with atomic fact augmentation and lookahead search. The agent extracts task-critical "atomic facts" from interaction trajectories to dynamically augment prompts for action proposal, world model simulation, and state-value estimation. Experimental results on TextFrozenLake and ALFWorld show improved performance and adaptability as the agent accumulates experience, demonstrating the effectiveness of fact-based abstraction and LLM simulation for online learning.
LLM agents can learn to plan more effectively in complex environments by distilling experiences into "atomic facts" that augment prompts, enabling lookahead search without fine-tuning.
Large Language Models (LLMs) are increasingly capable but often require significant guidance or extensive interaction history to perform effectively in complex, interactive environments. Existing methods may struggle with adapting to new information or efficiently utilizing past experiences for multi-step reasoning without fine-tuning. We introduce a novel LLM agent framework that enhances planning capabilities through in-context learning, facilitated by atomic fact augmentation and a recursive lookahead search. Our agent learns to extract task-critical ``atomic facts'' from its interaction trajectories. These facts dynamically augment the prompts provided to LLM-based components responsible for action proposal, latent world model simulation, and state-value estimation. Planning is performed via a depth-limited lookahead search, where the LLM simulates potential trajectories and evaluates their outcomes, guided by the accumulated facts and interaction history. This approach allows the agent to improve its understanding and decision-making online, leveraging its experience to refine its behavior without weight updates. We provide a theoretical motivation linking performance to the quality of fact-based abstraction and LLM simulation accuracy. Empirically, our agent demonstrates improved performance and adaptability on challenging interactive tasks, achieving more optimal behavior as it accumulates experience, showcased in tasks such as TextFrozenLake and ALFWorld.
