Search papers, labs, and topics across Lattice.
This paper introduces Seeker, a novel method that learns to focus policy inputs on regions of interest (ROIs) based solely on action supervision, eliminating the need for external spatial labels. By iteratively updating a query with visual evidence derived from actions, Seeker generates progression-aware ROIs that enhance data efficiency and robustness in visuomotor learning. Experimental results demonstrate significant improvements in task success rates on real robots, with in-domain success increasing from 48.3% to 76.7% and success under varying conditions rising from 20.0% to 60.0%.
Action-derived visual attention can boost robot task success rates by over 28% without relying on external labels.
Visual bottlenecks that focus policy inputs on regions of interest (ROIs) can improve data-efficient visuomotor learning by separating where to look from how to act. Many ROI interfaces rely on external spatial labels, such as gaze, object classes, or affordance annotations. Label-free alternatives often derive crops from trajectories by detecting gripper or motion events and centering a fixed crop at the projected end-effector. Such action-derived crops are useful spatial priors that require no additional labels, but they encode fixed choices about event timing, proxy points, and crop scale. When the visual evidence needed for control lies away from the end-effector or changes continuously with task progress, these crops can become misaligned. We propose Seeker, a task- and state-conditioned readout that learns attention from action. Starting from frozen DINOv3 features, Seeker iteratively updates a query with gathered visual evidence, producing progression-aware ROIs solely from action supervision. The learned ROI serves as a spatial interface for RGB cropping, mask-guided background augmentation, and point-cloud filtering. In simulation and the real world, Seeker improves data efficiency and robustness over no-crop, augmentation, and action-derived crop baselines. On real robots, Seeker raises average in-domain success from the best baseline's 48.3% to 76.7% and success under lighting/background shifts from 20.0% to 60.0%.
