Search papers, labs, and topics across Lattice.
This paper introduces ObjectStream, a framework that leverages latent objects as memory anchors for streaming video understanding, allowing models to efficiently retain and organize visual evidence over time. By linking spatially coherent latent objects across frames and maintaining their histories within a bounded memory budget, ObjectStream enhances the reasoning capabilities of existing Video Large Language Models (Video-LLMs) without altering their underlying architecture. Experimental results demonstrate that ObjectStream significantly improves performance on both online and offline benchmarks while drastically reducing memory usage and processing time.
Latent objects can revolutionize how Video-LLMs manage memory, achieving a 10-point performance boost while slashing memory usage by 50%.
Streaming video understanding requires models to continuously retain useful visual evidence before future questions are known. Existing approaches primarily manage the growing visual context according to token importance, temporal redundancy, or segment-level relevance, but rarely organize evidence around objects that persist and evolve over time. Thus, in this paper, we introduce ObjectStream, a training-free framework that treats latent objects as memory anchors for streaming video understanding. ObjectStream induces spatially coherent latent objects directly from frozen Video-LLM representations, links them across frames into persistent anchors, and maintains their histories under a bounded memory budget, without requiring external object detectors or segmentation models. Built on these anchors, ObjectStream preserves three complementary forms of evidence: persistent object histories, transient object changes, and recent visual context. This design enables existing Video Large Language Models (Video-LLMs) to reason over object identities, interactions, and state changes while leaving the underlying model unchanged. Extensive experiments on online streaming and offline long-video benchmarks demonstrate both effectiveness and efficiency. In online streaming evaluation, ObjectStream improves Qwen2.5-VL-7B by 10.0 points on OVO-Bench Real-Time Visual Perception, while reducing peak GPU mem-ory and TTFT by approximately 50%. On offline long-video benchmarks, it surpasses the full-token baseline while discarding 82.5% of visual tokens. These results highlight latent objects as a practical and effective organizing principle for compact streaming video memory.
