Search papers, labs, and topics across Lattice.
This paper introduces Tactile-WAM, a touch-aware World Action Model that incorporates a Tactile Asymmetric Attention Mechanism (TAAM) to enhance robot decision-making in contact-rich manipulation tasks. By addressing the issue of tactile pollution, where irrelevant tactile information degrades visual predictions, Tactile-WAM effectively separates tactile data from visual inputs while still utilizing it for action generation. The method demonstrates a significant improvement in performance, achieving a 38.9% increase in mean success rates and an 86% boost on tasks requiring precise contact handling.
Tactile-WAM achieves a remarkable 38.9% improvement in action success rates by effectively managing tactile information in robot decision-making.
World Action Models (WAMs) generate actions together with predicted futures, offering a powerful interface for robot decision making. In contact-rich manipulation, however, visually plausible futures can be physically incomplete: insertion, assembly, search, and reorientation often depend on slip, jamming, contact normals, or small alignment errors that are weakly visible or hidden in RGB. A natural solution is to predict future tactile states, however, we identify tactile pollution, a failure mode where unconstrained tactile-token injection degrades video and action prediction by forcing a visual dynamics model to absorb sparse, local, event-driven contact signals. To address this, we propose Tactile-WAM, a touch-aware WAM with a Tactile Asymmetric Attention Mechanism (TAAM). TAAM combines a VideoClean mask, which blocks video-query access to tactile key/value tokens while preserving action-query access, with a touch-aware bias for action attention. The VideoClean mask protects visual prediction while keeping contact information available for action generation; the touch-aware bias is derived from predicted touch changes and modulates action attention to tactile tokens during denoising. On ManiFeel, Tactile-WAM improves the mean success rate by 38.9% overall and by 86% on contact-rich tasks.
