Search papers, labs, and topics across Lattice.
Tac2Real is introduced as a lightweight visuotactile simulation framework designed for efficient online RL training, integrating the PNCG-IPC method with a multi-GPU parallel architecture to generate marker displacement fields rapidly. To ensure sim-to-real transfer, the TacAlign approach is proposed to mitigate both structured and stochastic domain gaps. Evaluation on a peg insertion task demonstrates high zero-shot transfer success rates in the real world, validating the framework's effectiveness.
Zero-shot visuotactile policies trained in a fast, parallelized simulator can directly control real robots in contact-rich tasks.
Visuotactile sensors are indispensable for contact-rich robotic manipulation tasks. However, policy learning with tactile feedback in simulation, especially for online reinforcement learning (RL), remains a critical challenge, as it demands a delicate balance between physics fidelity and computational efficiency. To address this challenge, we present Tac2Real, a lightweight visuotactile simulation framework designed to enable efficient online RL training. Tac2Real integrates the Preconditioned Nonlinear Conjugate Gradient Incremental Potential Contact (PNCG-IPC) method with a multi-node, multi-GPU high-throughput parallel simulation architecture, which can generate marker displacement fields at interactive rates. Meanwhile, we propose a systematic approach, TacAlign, to narrow both structured and stochastic sources of domain gap, ensuring a reliable zero-shot sim-to-real transfer. We further evaluate Tac2Real on the contact-rich peg insertion task. The zero-shot transfer results achieve a high success rate in the real-world scenario, verifying the effectiveness and robustness of our framework. The project page is: https://ningyurichard.github.io/tac2real-project-page/
