Search papers, labs, and topics across Lattice.
This paper introduces the SkillMemo framework, which enhances embodied visuomotor models by decomposing long-horizon demonstrations into atomic skills and integrating these into a dynamic episodic memory bank. By employing an expert-guided trajectory segmentation module based on a Mixture-of-Experts architecture, SkillMemo effectively partitions trajectories into distinct skill primitives, allowing for improved compositional generalization in out-of-distribution scenarios. Experimental results show that SkillMemo significantly boosts the performance of both Diffusion Policy and Vision-Language-Action models, achieving state-of-the-art results on robotic manipulation tasks and demonstrating strong adaptability to unseen configurations.
SkillMemo transforms robotic manipulation by enabling models to leverage reusable skill structures, leading to unprecedented compositional generalization in complex tasks.
Embodied visuomotor models, including Diffusion Policy (DP) and Vision-Language-Action (VLA) models, have demonstrated promising performance on robotic manipulation benchmarks. However, their potential remains fundamentally constrained by the scarcity of large-scale embodied trajectory datasets, leading to insufficient compositional generalization in out-of-distribution (OOD) scenarios with limited capability to capture reusable skill structures. To address this limitation, we propose Skill-Based Memory (SkillMemo) framework that implicitly decomposes long-horizon demonstrations into latent atomic skills and integrates skill-level features into a dynamic episodic memory bank for solving compositional tasks. Specifically, we first introduce an expert-guided trajectory segmentation module built upon a Mixture-of-Experts (MoE) architecture, which implicitly partitions trajectories into distinct skill primitives represented by learned gating coefficients. We further design a skill-level episodic memory architecture that stores compact skill representations as retrievable key-value pairs. During inference, the memory bank retrieves the most relevant skill primitives which are subsequently fused with the model's current gating distribution, providing a robust contextual prior to refine action predictions. Extensive experiments on the simulation benchmark and real-world manipulation tasks demonstrate that SkillMemo consistently enhances both DP and VLA backbones, achieving state-of-the-art performance and outperforming $\pi_{0.5}$, while exhibiting strong compositional generalization to unseen task configurations.
Tsinghua AI