Search papers, labs, and topics across Lattice.
This paper introduces MemoryCPT, an end-to-end trainable memory framework designed for long-horizon LLM agents, addressing the inefficiencies of existing memory systems that often rely on heuristic methods and excessive context passing. The framework comprises two main components: Query-agnostic Distillation (QAD) for compact memory construction and Query-aware Retrieval and Summarization (QAR) that utilizes reciprocal rank fusion and a LoRA-based summarizer optimized for cost-efficiency. Experimental results demonstrate that MemoryCPT significantly enhances the cost-performance trade-off compared to existing methods, as evidenced by evaluations on LoCoMo and LongMemEval datasets.
MemoryCPT achieves a superior cost-performance trade-off for LLM agents by intelligently managing memory without overwhelming downstream models with context.
Long-horizon LLM agents require memory systems that recover useful evidence from large interaction histories without passing excessive context to downstream models. Existing memory pipelines often rely on hand-crafted heuristics and repeated LLM calls, which can introduce redundant context and high inference cost. We propose MemoryCPT, an end-to-end trainable agent memory pipeline that spans offline memory construction and online query-conditioned context generation. MemoryCPT consists of two stages: Query-agnostic Distillation (QAD), which distills a modular memory-construction pipeline into a compact model using explicit reasoning traces; and Query-aware Retrieval and Summarization (QAR), which combines reciprocal rank fusion (RRF) with a LoRA-based summarizer trained via Group Relative Policy Optimization (GRPO) under a cost-aware reward. We further introduce Quality per Cost (QPC) to quantify answer quality per unit inference cost. Experiments on LoCoMo and LongMemEval show that MemoryCPT improves the cost-performance trade-off over the evaluated baselines, while ablation and sensitivity analyses characterize the contributions of its components and the effects of key design choices.
