Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces a streaming retrieval architecture for edge cameras that filters redundant frames using an on-device epsilon-net to retain only semantically novel frames, creating a denoised embedding index. This approach mitigates the problem of redundant frames degrading cross-modal retrieval performance in continuous video streams. Results show that this single-pass streaming filter outperforms offline alternatives across various vision-language models and datasets, achieving 45.6% Hit@5 with an 8M on-device encoder at 2.7 mW.
Edge devices can achieve state-of-the-art cross-modal retrieval by throwing away most of the data.
Always-on edge cameras generate continuous video streams where redundant frames degrade cross-modal retrieval by crowding correct results out of top-k search. This paper presents a streaming retrieval architecture: an on-device epsilon-net filter retains only semantically novel frames, building a denoised embedding index; a cross-modal adapter and cloud re-ranker compensate for the compact encoder's weak alignment. A single-pass streaming filter outperforms offline alternatives (k-means, farthest-point, uniform, random) across eight vision-language models (8M-632M) on two egocentric datasets (AEA, EPIC-KITCHENS). Combined, the architecture reaches 45.6% Hit@5 on held-out data using an 8M on-device encoder at an estimated 2.7 mW.