Search papers, labs, and topics across Lattice.
This paper introduces NativeMEM, a novel Vision-Language-Action (VLA) policy that employs Native Memory Compression to efficiently manage long-term visual histories in robotic manipulation tasks. By utilizing an innovative memory encoding scheme that compresses historical frames into single tokens, NativeMEM allows for real-time updates without the need for external memory management, significantly enhancing both the success rates and efficiency of pretrained VLA models. The results demonstrate a substantial improvement in performance, with success rates increasing from 32.4% to 84.0% in simulations and reaching 98.7% on real robots, while also requiring only 20% of the training data compared to previous methods.
NativeMEM achieves a staggering success rate of 98.7% on real robots by compressing visual histories into single tokens, revolutionizing long-horizon robotic manipulation.
How can pretrained Vision-Language-Action (VLA) models retain long-horizon visual histories with high-frequency updates without sacrificing efficiency? Existing approaches rely on external memory management, which restrains either the memory horizon or the reactiveness of pretrained policies. To this end, we present NativeMEM, a VLA policy that features long-term and real-time updated memory. At its core is an efficient memory encoding scheme, Native Memory Compression, which repurposes the VLA's own vision encoder to compress each historical frame from each camera view into a single token. Appended to the input sequence, these memory tokens enable the pretrained VLA to attend over long-term history with negligible latency overhead, requiring neither an external planner nor a freshly initialized memory module. To align the memory tokens with the pretrained policy, we first develop a generic memory tokenizer under the supervision of a frozen VLA on memory-demanding data, and then unfreeze the VLA for task-specific fine-tuning. NativeMEM consistently outperforms prior methods, boosting success rates from 32.4% to 84.0% in simulation and up to 98.7% on real robots, while maintaining low inference latency and GPU memory usage. Notably, NativeMEM exhibits high data efficiency by achieving competitive results with prior arts using only 20% of the training data.
