Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces DexDirect, a novel method for collecting dexterous manipulation demonstrations using direct kinesthetic guidance of a robot arm, which significantly enhances the efficiency of demonstration collection. By allowing operators to physically manipulate a 6-DoF robot arm while a webcam retargets their hand movements onto a 13-DoF dexterous robot hand, the approach yields a dramatic increase in successful demonstrations—17.2 times more than vision-based methods and 3.2 times more than pose-tracking methods. User studies indicate that while DexDirect increases physical demand, it substantially reduces cognitive workload, leading to a 90% success rate in a subsequent cube pick-and-place task when using a diffusion policy trained on the collected demonstrations.
Operators can collect 17.2 times more successful dexterous manipulation demonstrations with DexDirect compared to traditional methods, all while reducing mental workload.
Scalable collection of dexterous manipulation demonstrations remains a major bottleneck for robot learning. High-fidelity interfaces often require costly hardware and extensive setup, while low-setup, low cost alternatives tend to provide less precise control and impose greater cognitive workload on operators. We present DexDirect, a direct kinesthetic arm guidance for efficient dexterous demonstration collection. The operator drags a 6-DoF gravity-compensated robot arm directly by a handle, while a single webcam retargets operator's other hand onto a 16 joints 13-DoF dexterous robot hand. User studies suggest DexDirect collects 17.2x and 3.2x more successful demonstrations compared to purely vision (AnyTeleop) and pose-tracking (TeleDex) baselines. An adapted NASA-TLX shows DexDirect greatly reduces mental demand, effort, and frustration, despite raising physical demand. A diffusion policy trained on DexDirect demonstrations reaches a 90% success rate on a cube pick-and-place task. These results suggest that direct kinesthetic arm guidance combined with vision-based hand retargeting provides an efficient low-setup and scalable interface for collecting dexterous manipulation demonstrations