Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ViHaTeleop, a lightweight and affordable visual-haptic teleoperation system designed to enhance dexterous manipulation learning through improved demonstration quality. By integrating SLAM-based wrist tracking, camera-based hand tracking, and vibrotactile feedback, the system was tested across various contact-critical tasks, revealing that haptic feedback significantly increased success rates and subjective user confidence. The findings suggest that tactile cues can substantially improve performance in manipulation tasks, with a notable increase of 17 percentage points in success rates for peg-in-hole tasks when using visual-tactile policies.
Haptic feedback can boost success rates in dexterous manipulation tasks by up to 15.6 percentage points, transforming how we approach learning from demonstration.
Learning from demonstration is a promising approach for dexterous manipulation, but collecting high-quality contact-critical demonstrations remains difficult with low-cost teleoperation hardware. We present ViHaTeleop, a lightweight (0.7 kg), low-cost (\$550) visual-haptic teleoperation system with SLAM-based wrist tracking, camera-based hand tracking, and finger-wise vibrotactile feedback through Linear Resonant Actuators (LRA). The system includes several design choices (LED illumination, fisheye hand camera, and tactile-aware retargeting constraints) and is deployed on Franka + LEAP Hand + 9DTact in both real and simulated environments. Under matched with/without-haptic conditions with nine participants across six contact-critical tasks, haptics improved success rates across all tasks (+2.2 to +15.6 percentage points), while completion-time effects were task-dependent. Subjective ratings showed significant gains in contact clarity and grasp confidence in both simulation and real-world settings (Wilcoxon signed-rank, $p<0.05$). We also integrate a lightweight depth-camera-based tactile proxy in Isaac Sim, enabling a full pipeline from multi-modal demonstration collection to visual-tactile policy training. Preliminary downstream validation by training visual-tactile policies from collected demonstrations shows tactile cues benefit contact-critical subtasks (peg-in-hole: +17 percentage points over vision-only).