Search papers, labs, and topics across Lattice.
This paper introduces KEMO, a memory framework designed to enhance long-horizon robot manipulation by selectively preserving keyframes that correspond to task-relevant state changes. By integrating robot kinematics with visual filtering, KEMO encodes these keyframes into compact memory tokens and combines them with visual features using cross-attention and gated residual fusion. The approach significantly improves task success rates and stage completion rates in real-world dual-arm manipulation tasks, demonstrating a 23.6% increase in Task Success Rate and a 34.1% increase in Stage Completion Rate compared to memory-free baselines.
KEMO's event-driven keyframe memory boosts robot manipulation performance by over 23% by intelligently retaining only the most relevant task information.
Long-horizon robot manipulation remains challenging because similar observations may occur at different execution stages, while the appropriate action depends on previously completed operations. Memory can address this ambiguity by enabling policies to infer task progress from execution history. However, existing memory-augmented approaches often either retain dense histories that require compression or rely primarily on recent context that may discard earlier task-relevant events. In this work, we propose propose KEMO, a lightweight plug-in memory framework that automatically selectively preserves keyframes associated with task-relevant state changes for VLA policies. KEMO combines robot kinematics with visual filtering to detect events, encodes the selected keyframes as compact temporally ordered memory tokens, and integrates them with current visual features through cross-attention and gated residual fusion for VLA training. The detected events also define higher-weight training samples near critical transitions. We evaluate KEMO on various real-world dual-arm manipulation tasks spanning 2 to 6 scored subtasks, and trajectory length ranging from 830 steps to 2846 execution steps (durations from 28 to 95 seconds). Compared with the memory-free baseline (e.g., $蟺_{0.5}$), KEMO improves aggregate Task Success Rate by 23.6\% and Stage Completion Rate by 34.1\%. Ablations show that event-driven keyframe selection outperforms uniform sampling and recent-frame retention, while the proposed gated fusion and keyframe-aligned loss weighting provide complementary gains.
