Search papers, labs, and topics across Lattice.
ReTac-ACT, a vision-tactile imitation learning policy, was developed to improve precision assembly in contact-rich environments where visual feedback is limited. The policy uses bidirectional cross-attention for visuo-tactile feature enhancement, a proprioception-conditioned gating network to increase tactile reliance during visual occlusion, and a tactile reconstruction objective to learn manipulation-relevant contact information. Experiments on the NIST Assembly Task Board M1 benchmark demonstrated that ReTac-ACT achieves 90% peg-in-hole success, outperforming vision-only and generalist baselines, even at industrial-grade 0.1mm clearance.
A robot can now achieve 90% success in peg-in-hole tasks, even with only 0.1mm clearance, by intelligently fusing vision and tactile feedback when visual occlusion occurs.
Precision assembly requires sub-millimeter corrections in contact-rich "last-millimeter" regions where visual feedback fails due to occlusion from the end-effector and workpiece. We present ReTac-ACT (Reconstruction-enhanced Tactile ACT), a vision-tactile imitation learning policy that addresses this challenge through three synergistic mechanisms: (i) bidirectional cross-attention enabling reciprocal visuo-tactile feature enhancement before fusion, (ii) a proprioception-conditioned gating network that dynamically elevates tactile reliance when visual occlusion occurs, and (iii) a tactile reconstruction objective enforcing learning of manipulation-relevant contact information rather than generic visual textures. Evaluated on the standardized NIST Assembly Task Board M1 benchmark, ReTac-ACT achieves 90% peg-in-hole success, substantially outperforming vision-only and generalist baseline methods, and maintains 80% success at industrial-grade 0.1mm clearance. Ablation studies validate that each architectural component is indispensable. The ReTac-ACT codebase and a vision-tactile demonstration dataset covering various clearance levels with both visual and tactile features will be released to support reproducible research.
