Search papers, labs, and topics across Lattice.
This paper introduces TacVerse, a comprehensive multi-sensor dataset and benchmark designed to enhance cross-sensor vision-based tactile perception in robotics. The dataset, comprising 106,800 tactile images from seven different vision-based tactile sensors (VBTSs), facilitates three key tasks: shape classification, grating classification, and force regression, while revealing that direct cross-sensor transfer significantly degrades performance. Notably, few-shot adaptation for force regression improves outcomes on unseen sensors, and pretraining with a Masked Autoencoder consistently enhances performance across tasks, highlighting the importance of data-efficient adaptation strategies in tackling sensor variability.
Cross-sensor transfer in tactile perception suffers a performance drop, but few-shot adaptation can significantly mitigate this issue.
Vision-based tactile sensors (VBTSs) enable robots to infer contact geometry and force-related cues by imaging deformation through an internal camera, yet generalisation across sensor designs remains poorly understood. We present TacVerse, a multi-sensor dataset and benchmark for cross-sensor vision-based tactile perception. The dataset contains 106,800 tactile images from seven VBTSs and supports three downstream tasks: shape classification, grating classification, and force regression. Experiments are conducted under three settings: within-sensor training, zero-shot cross-sensor transfer, and few-shot adaptation. Strong within-sensor performance across all tasks indicates that the collected tactile observations are informative for the target objectives. Direct cross-sensor transfer, however, leads to substantial degradation. Shape classification is comparatively robust, whereas grating classification and force regression are more sensitive to sensor shift. Few-shot adaptation for force regression consistently improves performance on unseen target sensors but does not fully close the gap to within-sensor upper bounds. A representation study further shows that MAE (Masked Autoencoder) pretraining provides the most consistent gains across tasks and sensors. TacVerse provides a controlled testbed for studying sensor shift, data-efficient adaptation, and self-supervised learning in tactile perception.
