Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces EasyMimic, a framework for imitation learning on low-cost robots using human video demonstrations. It extracts 3D hand trajectories from RGB videos, aligns them to the robot's gripper control space, and employs a hand visual augmentation strategy to bridge the human-to-robot domain gap. By co-training a model on processed human data and a small amount of robot data, EasyMimic achieves high performance on manipulation tasks with the LeRobot platform, reducing the need for extensive robot data collection.
Forget expensive robot data collection: EasyMimic lets robots rapidly learn manipulation skills from readily available human videos using a clever action alignment and co-training approach.
Robot imitation learning is often hindered by the high cost of collecting large-scale, real-world data. This challenge is especially significant for low-cost robots designed for home use, as they must be both user-friendly and affordable. To address this, we propose the EasyMimic framework, a low-cost and replicable solution that enables robots to quickly learn manipulation policies from human video demonstrations captured with standard RGB cameras. Our method first extracts 3D hand trajectories from the videos. An action alignment module then maps these trajectories to the gripper control space of a low-cost robot. To bridge the human-to-robot domain gap, we introduce a simple and user-friendly hand visual augmentation strategy. We then use a co-training method, fine-tuning a model on both the processed human data and a small amount of robot data, enabling rapid adaptation to new tasks. Experiments on the low-cost LeRobot platform demonstrate that EasyMimic achieves high performance across various manipulation tasks. It significantly reduces the reliance on expensive robot data collection, offering a practical path for bringing intelligent robots into homes. Project website: https://zt375356.github.io/EasyMimic-Project/.