Search papers, labs, and topics across Lattice.
This paper introduces RoboTacDex, a comprehensive dataset designed to enhance humanoid robot manipulation through large-scale, multi-modal demonstration trajectories. The dataset includes 6,000 trajectories across 19 tasks and 23 skills, featuring multi-view RGB and depth data, tactile feedback, and semantic annotations, all collected using an advanced multi-camera synchronization system. Evaluation of various imitation learning models on RoboTacDex shows promising generalization capabilities, highlighting the dataset's potential to advance robotic dexterity in complex, human-like tasks.
RoboTacDex reveals that a diverse dataset of 6,000 trajectories can significantly improve humanoid robot manipulation across complex tasks.
In the field of robot learning, large-scale and diverse demonstration trajectories provide the fundamental basis for enhancing robotic manipulation ability. We introduce RoboTacDex, a large, multi-modal, and diverse dataset of dexterous manipulation behaviors performed with a humanoid robot. Built on the publicly accessible humanoid robot Unitree G1, RoboTacDex consists of 6k trajectories covering 19 tasks, 23 skills, and interactions with 22 objects. RoboTacDex provides comprehensive records including multi-view RGB and depth information, tactile feedback, and detailed semantic annotations. Furthermore, the dataset features a variety of relatively challenging tasks that can only be completed by dual arms and dexterous hands, aiming to mimic human-like operational logic and simulate real-world manipulation complexity. To ensure data collection quality, we develop an improved multi-camera synchronization system to enable millisecond data synchronization and recording of modalities. In our experiments, we evaluate three representative imitation learning models on our dataset, analyzing their performance as well as their respective strengths and limitations across different task categories. Successful trial results and a moderate level of generalization capabilities across a suite of tasks indicate the effectiveness and diversity of the collected dataset. Our dataset will be open-sourced soon.
