Search papers, labs, and topics across Lattice.
HydroShear, a novel non-holonomic hydroelastic tactile simulator, models stick-slip transitions, path-dependent force/shear buildup, and full SE(3) object-sensor interactions using Signed Distance Functions to track indenter displacements. This approach generates computationally efficient force fields from arbitrary geometries, agnostic to the physics engine. Experiments with GelSight Minis demonstrate that HydroShear enables zero-shot sim-to-real transfer of RL policies, achieving a 93% average success rate across four contact-rich tasks, significantly outperforming policies trained on tactile images or alternative shear simulation methods.
Achieve 93% success in zero-shot sim-to-real transfer for contact-rich tasks by realistically simulating tactile shear, blowing away prior art.
In this paper, we address the problem of tactile sim-to-real policy transfer for contact-rich tasks. Existing methods primarily focus on vision-based sensors and emphasize image rendering quality while providing overly simplistic models of force and shear. Consequently, these models exhibit a large sim-to-real gap for many dexterous tasks. Here, we present HydroShear, a non-holonomic hydroelastic tactile simulator that advances the state-of-the-art by modeling: a) stick-slip transitions, b) path-dependent force and shear build up, and c) full SE(3) object-sensor interactions. HydroShear extends hydroelastic contact models using Signed Distance Functions (SDFs) to track the displacements of the on-surface points of an indenter during physical interaction with the sensor membrane. Our approach generates physics-based, computationally efficient force fields from arbitrary watertight geometries while remaining agnostic to the underlying physics engine. In experiments with GelSight Minis, HydroShear more faithfully reproduces real tactile shear compared to existing methods. This fidelity enables zero-shot sim-to-real transfer of reinforcement learning policies across four tasks: peg insertion, bin packing, book shelving for insertion, and drawer pulling for fine gripper control under slip. Our method achieves a 93% average success rate, outperforming policies trained on tactile images (34%) and alternative shear simulation methods (58%-61%).
