Search papers, labs, and topics across Lattice.
This paper introduces the RCT dataset, a comprehensive collection of 29,279 tactile frames gathered by robots interacting with 122 industrial materials, aimed at enhancing tactile generalization in robotic manipulation. The dataset enables rigorous evaluation of tactile representations across various materials and contact conditions, revealing that removing contact-sequence overlap significantly hampers performance, with Recall@1 dropping by 17.7 percentage points. Notably, the findings highlight the critical challenge of generalizing tactile perception to novel materials, underscoring the limitations of current models in this domain.
Tactile representations in robots struggle to generalize, with performance dropping to just 25.1% Recall@1 when faced with unseen materials.
For robots manipulating open-world objects, tactile representations must generalize to unseen materials. We introduce RCT (Robotic Contact Tactile), a robot-collected touch-vision-language dataset with 29,279 tactile frames from full robot presses on 122 industrial reference materials in 7 categories, recorded with three DIGIT sensors at multiple contact positions. RCT preserves each press as a contact sequence, enabling held-out evaluation across materials, categories, sensors, contact positions, and contact sequences. Frames from one press are strongly correlated: frame-random splits can place near-duplicate observations of the same physical interaction in both training and test. With the encoder held fixed, removing contact-sequence overlap reduces tactile-to-text Recall@1 by 17.7 percentage points. When materials are additionally held out at training time, performance drops sharply, leaving held-out-material Recall@1 at 25.1 +/- 6.1% averaged over three held-out draws. The public TVL/HCT split shows the same structure: every test contact sequence appears in training, and raw-pixel nearest neighbors recover the correct sequence in 98.3% of cases. Uniformly sampling a press improves contrastive training, and RCT-trained embeddings improve category probes on unseen materials. RCT makes contact-sequence-aware, held-out-material evaluation reproducible and exposes novel-material generalization as a central challenge for robotic tactile perception. The RCT dataset is open-sourced at https://faerber-lab.github.io/RCT/
