Search papers, labs, and topics across Lattice.
GROVE is a novel framework that integrates causal memory growth from continuous video streams, enabling a wearable assistant to both recall visual history and proactively assess its relevance to current situations. By consolidating fine-grained perceptual evidence into time-stamped moments and coherent episodes, GROVE allows for efficient retrieval through scale-native skills tailored for various tasks. The framework outperforms existing methods across multiple benchmarks, demonstrating that its unique temporal strata and retrieval mechanisms significantly enhance performance, particularly when evidence spans multiple days.
GROVE achieves state-of-the-art performance in memory retrieval for wearable assistants by seamlessly integrating reactive and proactive memory access from continuous video streams.
A wearable assistant should both answer questions about its visual history and recognize when that history is useful to the present situation. Existing video-memory systems primarily support question-conditioned recall, whereas proactive assistants typically use separate memory and control mechanisms. We introduce GROVE, a training-free framework that supports both behaviors with one memory grown causally from a continuous video stream. GROVE retains fine-grained perceptual evidence and incrementally consolidates it into time-stamped moments, coherent episodes, and recurring cross-day patterns. Each stratum is paired with a scale-native retrieval skill for locating an observation, replaying an activity, or traversing long-range regularities. Reactive QA and proactive assistance share this memory and access interface, differing in whether retrieval is initiated by a user query or the current situation. Across multiple benchmarks including the challenging MM-lifelong and EgoServe, GROVE achieves the best results among the compared methods. Controlled ablations show that the temporal strata and their access skills are complementary, with patterns providing the largest benefit when evidence spans multiple days. Code will be available at https://github.com/SitongGong/GROVE.
NVIDIA