Search papers, labs, and topics across Lattice.
This paper addresses the challenge of blind grasping with dexterous robotic hands by proposing a novel framework that bridges the tactile sim-to-real gap. The authors introduce a Real2Sim tactile calibration pipeline, a layout-aware tactile encoder, and a diffusion-based policy aggregation method to enhance the expressiveness of sparse tactile signals and improve generalization to unseen objects. The resulting tactile-only policy achieves a 27% success rate in real-world grasping tasks across 20 objects, demonstrating significant advancements in tactile manipulation without relying on visual input or prior demonstrations.
Achieving a 27% real-world grasp success rate for blind grasping with dexterous hands using only tactile feedback marks a significant leap in robotic manipulation capabilities.
Blind grasping with a dexterous hand is a crucial manipulation capability. Nevertheless, learning such tactile-only policies for real robots remains challenging due to the tactile sim-to-real gap and the limited expressiveness of sparse tactile signals. To bridge this gap, we propose a framework for tactile-only blind grasping that is deployable on a physical multi-fingered robotic hand. Our approach combines three key components. First, we introduce a Real2Sim tactile calibration pipeline that constructs a contact-calibrated digital-twin simulator capable of reproducing real tactile signals. Second, we improve the expressiveness of sparse tactile observations using a layout-aware tactile encoder, which incorporates sensor-geometry priors through self-supervised pretraining. Third, to improve generalization to unseen objects, we train object-specific reinforcement-learning experts in the calibrated simulator and aggregate their successful grasp trajectories into a tactile-conditioned Diffusion Policy. We evaluate our method on a physical LEAP Hand equipped with distributed tactile sensing across 10 seen and 10 unseen objects. The deployed policy achieves a 27\% real-world grasp success rate across all 20 objects, without real-world grasping demonstrations or visual input. Simulation ablations show that layout-aware tactile pretraining improves grasping performance, while sensing-level evaluations confirm that Real2Sim calibration increases the consistency of tactile contact events between simulation and hardware. Together, these results suggest that contact-event calibration, geometry-aware tactile representation learning, and diffusion-based policy aggregation provide an effective path toward tactile-only blind grasping on real dexterous robotic hands. Project page:Dex-Blind-Grasp.github.io.
