Search papers, labs, and topics across Lattice.
The paper introduces TA-Mem, a tool-augmented autonomous memory retrieval framework for LLMs designed to address the context window limitations in long-term conversational question answering. TA-Mem employs a memory extraction LLM agent for adaptive chunking and structured note creation, a multi-indexed memory database supporting key-based lookup and similarity retrieval, and a tool-augmented retrieval agent that autonomously explores memory using database tools. Evaluated on the LoCoMo dataset, TA-Mem significantly outperforms existing baselines, demonstrating its adaptivity through tool use analysis across various question types.
LLMs can now autonomously retrieve relevant memories from a database using specialized tools, significantly improving performance on long-term conversational question answering.
Large Language Model (LLM) has exhibited strong reasoning ability in text-based contexts across various domains, yet the limitation of context window poses challenges for the model on long-range inference tasks and necessitates a memory storage system. While many current storage approaches have been proposed with episodic notes and graph representations of memory, retrieval methods still primarily rely on predefined workflows or static similarity top-k over embeddings. To address this inflexibility, we introduced a novel tool-augmented autonomous memory retrieval framework (TA-Mem), which contains: (1) a memory extraction LLM agent which is prompted to adaptively chuck an input into sub-context based on semantic correlation, and extract information into structured notes, (2) a multi-indexed memory database designed for different types of query methods including both key-based lookup and similarity-based retrieval, (3) a tool-augmented memory retrieval agent which explores the memory autonomously by selecting appropriate tools provided by the database based on the user input, and decides whether to proceed to the next iteration or finalizing the response after reasoning on the fetched memories. The TA-Mem is evaluated on the LoCoMo dataset, achieving significant performance improvements over existing baseline approaches. In addition, an analysis of tool use across different question types also demonstrates the adaptivity of the proposed method.
