Search papers, labs, and topics across Lattice.
This paper introduces RoboMME-Interference, a benchmark designed to evaluate robot memory performance in the presence of interference from unrelated sessions. The study reveals that while memory-augmented models show improved accuracy when provided with relevant historical data, their performance significantly deteriorates as the number of distracting sessions increases. These findings highlight a critical gap in current robot memory systems, emphasizing the need for advancements in long-context memory and interference robustness for real-world applications.
Current robot memory systems fail to maintain accuracy under interference, with performance dropping sharply as unrelated sessions accumulate.
Robots deployed in realistic settings will accumulate experience across many sessions and tasks over their deployment. The robot's tasks may often require it to remember information from multiple sessions ago, making long-context robot memory important for real-world deployments. However, most robot-memory benchmarks today are based on single episodes or a short context. To measure how current robot memory systems perform on longer sessions with more distractions, we introduce RoboMME-Interference, a cross-session benchmark built on RoboMME. For each query episode, we construct a session history using the query's relevant prior demonstration followed by a controlled number of unrelated sessions, which we provide to the VLA as memory and measure accuracy. Running RoboMME's released memory-augmented $蟺_{0.5}$ variants unmodified through this benchmark, we find that while perceptual memory variants improve success when given the history without any distractors, they decay strongly and steadily as unrelated sessions accumulate. With this release, we emphasize the importance of long-context memory and robustness to interference and show that current systems largely fail on such capabilities. The project page, videos, code, and data are at https://robotmemorybench.com.
