Search papers, labs, and topics across Lattice.
The paper introduces MemoAct, a hierarchical memory-augmented visuomotor policy inspired by the Atkinson-Shiffrin model, using lossless short-term memory for precise task state tracking and compressed long-term memory for robust long-horizon retention. They also introduce MemoryRTBench, a new benchmark built on RoboTwin 2.0, to evaluate task state tracking and long-horizon retention in robotic policies. Experiments in simulation and the real world show MemoAct outperforms Markovian and history-aware baselines on the new benchmark.
Hierarchical memory, inspired by human cognition, beats standard approaches in robotic manipulation tasks requiring both precise tracking and long-term retention.
Memory-augmented robotic policies are essential in handling memory-dependent tasks. However, existing approaches typically rely on simple observation window extensions, struggling to simultaneously achieve precise task state tracking and robust long-horizon retention. To overcome these challenges, inspired by the Atkinson-Shiffrin memory model, we propose MemoAct, a hierarchical memory-based policy that leverages distinct memory tiers to tackle specific bottlenecks. Specifically, lossless short-term memory ensures precise task state tracking, while compressed long-term memory enables robust long-horizon retention. To enrich the evaluation landscape, we construct MemoryRTBench based on RoboTwin 2.0, specifically tailored to assess policy capabilities in task state tracking and long-horizon retention. Extensive experiments across simulated and real-world scenarios demonstrate that MemoAct achieves superior performance compared to both existing Markovian baselines and history-aware policies. The project page is \href{https://tlf-tlf.github.io/MemoActPage/}{available}.
