Search papers, labs, and topics across Lattice.
This paper introduces the RE4 framework, which integrates principled theories of manipulation into imitation learning for object interaction tasks, enhancing both performance and interpretability. By employing a lightweight model-free pose estimation method and a manipulation mode-aware retrieval process, the authors achieve robust results across state-based and image-based benchmarks, including challenging low-data scenarios. The findings indicate that interpretable components can effectively facilitate the learning of complex manipulation skills, addressing a critical gap in current imitation learning approaches.
Transforming how we learn object interactions, RE4 combines interpretability with high performance in imitation learning, even in low-data environments.
Object interaction tasks have been a focus of advances in imitation learning. End-to-end methods, dominated by diffusion and flow-based variants have shown leaps in performance while sacrificing interpretability. Object-centric and pose-informed variants have had a role in learning from demonstration in manipulation tasks. In this paper, we revisit a few modern imitation learning benchmarks for object interactions, with the aim of composing a framework that repurposes principled theories of manipulation, preserving both performance and interpretability. For image observations, lightweight training is proposed for model-free pose estimation of the target object, using self-supervision over the demonstration data available for imitation learning. This information is then used to inform a manipulation mode-aware retrieval of a demonstration, a mode-aware transformation, a replan step that connects to the retrieval point while preserving mode constraints, and finally rolling out the transformed demonstration. These compose four key steps of the proposed RE4 framework, evaluated over state-based and image-based benchmarks in Push-T and Robomimic. An adversarial benchmark that evaluates sparse data regions of image-based Push-T showcases the robustness, further bolstered by indications from low-data regime experiments. The current work shows promise in using simple interpretable building blocks to learn manipulation skills.
