Search papers, labs, and topics across Lattice.
This paper introduces Multi-Resolution Tactile Sensing (MiTaS), a framework that integrates multiple tactile sensors with varying temporal resolutions to enhance robotic manipulation in contact-rich environments. By employing a novel architecture that combines modality-specific convolutional stems and transformer-based fusion, the authors demonstrate significant improvements in imitation learning tasks, achieving an average success rate of 80% compared to 31% for vision-only and 54% for visual-tactile baselines. The findings underscore the critical role of diverse tactile information in improving robotic performance, particularly when co-training with multi-tactile data.
Multi-resolution tactile sensing boosts robotic manipulation success rates to 80%, far surpassing traditional vision-only approaches.
Touch sensing is beneficial for solving a wide variety of manipulation tasks. While there exists a wide range of tactile sensors with different properties, exploiting the fusion of multiple heterogeneous tactile sensors to improve manipulation learning remains underexplored. We present Multi-Resolution Tactile Sensing (MiTaS), a representation framework that leverages multiple tactile sensors operating at different temporal resolutions in order to solve complex contact-rich manipulation tasks. We propose a novel architecture using modality-specific convolutional stems and transformer-based fusion that effectively fuses information from an RGB camera stream, a vision-based GelSight Mini sensor and a high-frequency event-based Evetac sensor. This multi-sensor representation then conditions a flow-matching policy for solving downstream tasks. Experimental results across five contact-rich manipulation tasks demonstrate the effectiveness of multi-resolution tactile features in imitation learning. MiTaS achieves an average success rate of 80 %, while vision-only (31 %) and visual-tactile (54 %) baselines cannot solve the task reliably. Co-training a visuo-tactile model with multi-tactile data boosts performance by over 10 \% in certain tasks, without having access to the Evetac sensor during policy evaluation. A detailed sensor-reading and attention analysis reveals the importance of different sensors throughout task execution, validating our multi-resolution tactile sensing approach. Project Page: http://mitas-touch.github.io.
