Search papers, labs, and topics across Lattice.
This paper introduces H-Tac, a comprehensive tactile-action dataset comprising 160 hours of egocentric human videos across over 300 tasks, addressing the limitations of existing small-scale tactile datasets. The authors propose Transferable Tactile Pre-Training (TTP), which utilizes a unified tactile and action space to enhance the transfer of human manipulation skills to robots, effectively modeling contact dynamics for improved performance. Experimental results show that TTP significantly boosts the generalization and fine-grained manipulation capabilities of robotic systems in both simulated and real-world environments.
TTP enables robots to learn dexterous manipulation from human tactile experiences, achieving unprecedented performance in complex tasks.
As an essential modality for dexterous and contact-rich tasks, tactile sensing provides precise force feedback that cannot be reliably inferred from vision. However, limited by hardware and data collection systems, existing datasets with tactility remain small in scale and narrow in contact coverage. Meanwhile, Vision-Language-Action (VLA) models with tactile modality are constrained on dynamics-agnostic post-training, which limits the performance ceiling on downstream tasks. In this paper, we present H-Tac, a large-scale tactile-action dataset with 160-hour egocentric human videos containing more than 300 tasks and 135k episodes. Building upon this, we propose Transferable Tactile Pre-Training (TTP), a system of tactile-based pre-training on human data for fine-grained robotic tasks. To bridge the gap between humans and robots, we use unified tactile and action spaces throughout the pre-training and post-training phases, preserving prior knowledge during human-to-robot transfer. By leveraging a tactile expert for future tactile prediction, our framework explicitly models the contact dynamics and precise physical interactions. Extensive experiments in simulation and on real robots demonstrate that our model achieves superior performance, exhibiting robust generalization and fine-grained manipulation capabilities. TTP paves the way for scalable tactile pre-training via human-to-robot transfer.