Search papers, labs, and topics across Lattice.
This paper introduces Tac-DINO, which addresses the limitations of current tactile learning by constructing a large-scale dataset of over 20,000 tactile contacts from 505 real-world objects. The authors develop a Vision-Tactile Patch Alignment (VTPA) method that enhances vision-tactile representation learning through local-to-global alignment, evaluated using a newly created Vis-Tac Holographic Matching Benchmark. The results show that VTPA significantly outperforms existing methods lacking alignment, highlighting the importance of integrating tactile signals for improved object interaction understanding.
Tactile learning can now leverage a novel dataset and alignment method to dramatically enhance vision-tactile representation, outperforming traditional approaches.
Touch is the primary medium through which humans interact with the environment. Currently, tactile learning mainly focuses on image-level pretraining or alignment. However, tactile signals correspond to local object contact, while research into scale alignment and holographic matching remains limited and proper datasets and benchmarks also lack. To bridge this gap, we first construct a data collection system to acquire a large-scale tactile dataset, with over 20 K tactile contacts from 505 real-world objects. Building on this dataset, we design a Vis-Tac Holographic Matching Benchmark to evaluate vision-tactile local-to-global alignment ability. Then we propose Vision-Tactile Patch Alignment (VTPA) methods for vision-tactile representation learning. Experiments demonstrate that these exceed the performance of methods without alignment and align with whole-object images.
