Search papers, labs, and topics across Lattice.
This paper introduces EgoMonth, the first month-level egocentric video understanding benchmark, comprising over 300 hours of first-person recordings from 20 participants. The benchmark aims to evaluate the ability of Multimodal Large Language Models (MLLMs) to maintain consistent memory across extended periods, revealing that even the best-performing models fall significantly short of human-level performance. The findings indicate that current MLLMs primarily act as lossy summarizers rather than effective long-term memory systems, underscoring the necessity for improved architectures in this domain.
Current MLLMs struggle with long-term memory, achieving only 71.8% accuracy on a benchmark where humans score 94.2%.
Recent advances in Multimodal Large Language Models (MLLMs) have led to substantial progress in video understanding, accompanied by a growing number of long video benchmarks. However, existing benchmarks rely predominantly on web-sourced videos that lack inter-clip spatiotemporal continuity, making it difficult to assess whether models can maintain consistent memory across days or weeks of real-world experience. We introduce EgoMonth, the first month-level egocentric video understanding benchmark. EgoMonth comprises over 300 hours of first-person daily-life recordings from 20 participants spanning 20 to 120 days, paired with 1,443 human-crafted multiple-choice question-answer pairs. We design a cognitively grounded 14-task evaluation framework organized into three hierarchical cognitive levels: Schema Consolidation, Episodic Indexing, and Cascading Reasoning. Evaluation of state-of-the-art open-source and closed-source MLLMs reveals that even the best-performing model, Gemini 2.5 Pro, achieves only 71.8% macro-average accuracy, remaining 22.4 percentage points below the corrected human baseline of 94.2%. Several models perform near or below the 25% chance level on tasks such as Route Reasoning, Cross-view Spatial Reasoning, and Direction Judgement, while even the strongest closed-source model remains substantially below human performance. These results indicate that current MLLMs function as lossy summarizers rather than faithful memorizers, highlighting the need for architectures with genuine long-term spatiotemporal memory.
