Search papers, labs, and topics across Lattice.
This review comprehensively analyzes vision-based tactile sensors (VBTSs) that transform contact-induced deformations into high-resolution images, significantly enhancing robotic perception and manipulation in contact-rich environments. By organizing VBTSs into a taxonomy based on design and functionality, and presenting a hierarchical framework for learning-based tactile intelligence, the authors elucidate the interplay between hardware, AI methods, and simulation. The paper also highlights key challenges and future directions, aiming to propel the integration of tactile sensing and learning in robotics.
Vision-based tactile sensors could revolutionize robotic interaction by providing high-resolution tactile data that enhances perception and manipulation capabilities.
Tactile sensing is essential for robots in contact-rich tasks, yet many tactile sensors still provide sparse, low-dimensional signals that do not capture sufficient information for complex robotic perception and interaction. Vision-based tactile sensors (VBTSs) offer a powerful alternative by con-verting contact-induced deformation of a soft interface into im-ages. The image-based formulation gives VBTSs high-resolution, information-rich tactile observations that enable complex robotic tasks. This review surveys the full VBTS pipeline and treats sensing hardware, learning methods, simulation, and datasets as an integrated sensing-and-learning system. We 1) organize representative VBTSs into a hardware taxonomy structured by deformable elastomer design, sensor size and shape, and optical system design to guide future sensor development; 2) present a hierarchical view of learning-based tactile intelligence from low-level signal understanding to task-level policies and foundation models; and 3) examine simulation platforms and tactile datasets as a scaling layer, together with sim-to-real transfer and cross-sensor adaptation for training, benchmarking, and deployment. Finally, we identify open challenges and future directions for VBTSs in robotics. By providing a holistic view of how hardware, AI architectures, simulation, and datasets interact, this review aims to advance tactile intelligence for contact-rich robotic tasks.
Tsinghua AI