Search papers, labs, and topics across Lattice.
This paper introduces Tactus, an innovative model for open-vocabulary object recognition using low-cost resistive pressure arrays, achieving competitive performance on the STAG benchmark. By leveraging a small dataset of 187 recordings and employing masked-autoencoder pretraining on a large corpus of unlabeled data, Tactus matches or exceeds the accuracy of traditional supervised CNNs without requiring a dedicated classifier head. The model's robustness is highlighted by its ability to handle paraphrased queries and maintain performance across diverse input frames, while also providing insights into its failure modes and limitations.
Tactus achieves state-of-the-art object recognition from pressure data alone, outperforming traditional methods without a trained classifier head.
Resistive pressure arrays are the cheapest and most widely shipped tactile sensors, yet tactile representation learning has concentrated on optical sensors that image a deforming gel. We present Tactus, an open model that answers text queries from pressure data alone: on the STAG benchmark (27 objects, held-out recordings), it reaches 0.771 +/- 0.062 top-1 over four runs (top-3 0.935), matching, and at best exceeding, the dataset's supervised closed-set CNN at 0.76, with no trained classifier head. The recipe is small-data: 187 training recordings, masked-autoencoder pretraining on 144k unlabeled same-sensor frames, and the sensor's own calibration affine, which recovered more accuracy than every architecture change combined. The released model's errors concentrate in a few contact-ambiguous classes, are uncorrelated with text-target geometry (Spearman rho<= 0.05 over 702 class pairs), and survive paraphrased and even bare-name queries within one point; two diverse frames recover 89% of eight-frame accuracy. Failures are reported with equal precision: cross-sensor pretraining pooling gave no gain, vision co-training degraded touch, and a mis-normalized input pipeline silently discarded 97% of the sensor's dynamic range while producing plausible intermediate results. Weights, code, and the memory layer the model plugs into are released openly.
