Search papers, labs, and topics across Lattice.
This paper introduces EgoCITE, a framework designed to enhance long-horizon egocentric memory by addressing the limitations of context-poor indexing and temporal intent in retrieval. By employing a multimodal approach to create self-contained memory indices and integrating semantic search with temporal relevance scoring, EgoCITE significantly improves the accuracy of question-answering tasks. The results demonstrate a 4.4-14.2% increase in answer accuracy and a 36x reduction in computational cost compared to traditional long-context LLM agents.
EgoCITE achieves a remarkable 36x reduction in cost while boosting answer accuracy by over 14% in egocentric memory tasks.
Long-horizon egocentric memory transforms continuous first-person video and audio into a searchable record of past experiences. We demonstrate two bottlenecks in existing systems: indices built from context-poor captions are unreliable for agentic search, while retrieval ignores a question's temporal intent. To address both bottlenecks, we introduce EgoCITE (Egocentric Context-augmented Indexing and Time-aware Evidence retrieval), a long-horizon agentic memory framework for egocentric QA. EgoCITE comprises three components. EgoScheme uses local multimodal context to turn fragmentary video captions and speech transcripts into self-contained atomic memory indices. EgoIndex organizes complementary action, activity, utterance, and conversation representations into searchable multi-view memory indices at multiple granularities. EgoRetrv combines semantic search with question-conditioned temporal relevance scoring and curation of retrieved evidence. We evaluate EgoCITE on EgoLifeQA, EgoMem, and EgoR1-Bench in terms of answer accuracy and target-event retrieval alignment. EgoCITE improves accuracy over agentic memory baselines by at least 4.4--14.2\% while achieving 36$\times$ lower cost than long-context LLM agents.
