Search papers, labs, and topics across Lattice.
The paper introduces a Dual-Criterion Frame Curator for efficient egocentric video curation using eye-tracking data from wearable devices. This curator leverages gaze fixation to filter for visual stability (quality) and pupil response to rank frames by novelty, addressing the redundancy and low-quality issues in egocentric video streams. Experiments on the Visual Experience Dataset (VEDB) demonstrate that curated frames at a 10% budget achieve classification performance comparable to the full stream, with task-dependent benefits from pupil-derived novelty.
Ditch the endless egocentric video: eye-tracking data lets you cherry-pick the 10% of frames that matter, matching full-stream performance without any training.
Always-on egocentric cameras are increasingly used as demonstrations for embodied robotics, imitation learning, and assistive AR, but the resulting video streams are dominated by redundant and low-quality frames. Under the storage and battery constraints of wearable devices, choosing which frames to keep is as important as how to learn from them. We observe that modern eye-tracking headsets provide a continuous, training-free side channel that decomposes into two complementary axes: gaze fixation captures visual stability (quality), while pupil response captures arousal-linked moments (novelty). We operationalize this insight as a Dual-Criterion Frame Curator that first gates frames by gaze quality and then ranks the survivors by pupil-derived novelty. On the Visual Experience Dataset (VEDB), curated frames at 10% budget match the classification performance of the full stream, and naive signal fusion consistently destroys both contributions. The benefit is task-dependent: pupil ranking improves activity recognition, while gaze-only selection already dominates for scene recognition, confirming that the two signals serve genuinely different roles. Our method requires no model inference and operates at capture time, offering a path toward efficient, always-on egocentric data curation.
