Track synthesis, materials and catalysis as it is published.
Search papers, labs, and topics across Lattice.
To overcome the entity-conflation and fragmentation issues inherent to chronological text summaries in multi-day video QA, the authors develop Grounded Entity Biographies (GEB), a long-video memory framework that clusters visually grounded instances of identical physical objects into persistent biographies. By coupling visual identity linking across disparate clips with episodic context, the system provides downstream models with explicit entity trajectories rather than isolated temporal snapshots. Across four long-form video benchmarks, GEB consistently outpaces existing memory systems, achieving 72.0% accuracy on EgoLifeQA (+4.4 percentage points over the prior state of the art).
Chronological text logs routinely confuse distinct physical objects across multi-day video, a failure mode resolved by explicitly clustering grounded visual instances into persistent entity biographies rather than relying on episodic text search.
Answering questions about long videos often requires connecting events involving the same objects across hours or days. Chronological descriptions and text-derived entities can leave physical identity unresolved: different objects may share a description, while observations of the same object remain disconnected across events. Retrieving relevant events therefore does not necessarily recover the "biography" of the particular entity a question concerns. To address this, we introduce Grounded Entity Biographies (GEB), a long-video memory framework that groups visually grounded observations of the same physical instance across clips into retrievable biographies while preserving the context of each moment. During question answering, the biography is retrieved alongside episodic evidence, allowing the model to follow an entity through events using identity links established during memory construction. Evaluations across four benchmarks, including day-long and week-long recordings, demonstrate improvements over prior memory frameworks in both multiple-choice and open-ended question answering. On EgoLifeQA, GEB achieves 72.0% accuracy, 4.4 percentage points above the best published result. Ablations show that grounded identity association and biography reading both contribute to the gains, which additional descriptions alone do not fully recover.