Search papers, labs, and topics across Lattice.
HiTac-WAM introduces a hierarchical tactile world action model that predicts future tactile states for robot manipulation by organizing contact states, deformation fields, and slip risks into a directed hierarchy. This model significantly enhances action planning and execution by using tactile forecasts and task-progress estimates to rank candidate actions, leading to improved success rates in real-world tasks. The results show a remarkable increase in the average success rate from 31.1% to 72.2%, demonstrating the model's effectiveness in contact-rich environments.
Hierarchical tactile modeling boosts robot manipulation success rates by over 40% in contact-rich tasks, revealing the power of structured tactile forecasting.
World action models jointly predict future visual observations and actions, whereas existing tactile-aware variants typically represent future touch as an image or latent stream without modeling the physical dependencies that organize tactile states hierarchically. We present HiTac-WAM, a hierarchical tactile world action model that forecasts a sequence of future tactile states for each candidate action chunk before execution. The forecast factorizes into contact state, a 3D deformation field, and slip risk, organized as a directed hierarchy in which each downstream stage is conditioned on stop-gradient signals from preceding stages. A directed attention mask allows tactile queries to attend to the video-action context of each candidate while preventing video and action queries from attending to tactile tokens. For planning, HiTac-WAM ranks candidate action chunks using tactile forecasts and task-progress estimates. For execution, the selected tactile forecast is retained as a reference; persistent discrepancies between predicted and observed tactile states trigger corrective replanning. HiTac-WAM achieves a mean contact F1 of 0.921; under matched training budgets, the directed hierarchy reduces 3D displacement L2 error by 17.6% relative to the deformation-only predictor and improves slip AUPRC by 60.4% relative to the slip-only predictor. Across chip grasping, blackboard erasing, and USB insertion, selection guided by the hierarchical forecasts increases the average real-robot success rate from 31.1% to 61.1%, while the full system attains 72.2%.
