Search papers, labs, and topics across Lattice.
This paper presents ReTouch, a vision-language-action model designed to enhance contact-rich dexterous manipulation in robots by integrating tactile predictions that are refined online using execution-time feedback. The model employs a Tactile-Patch Encoder to represent tactile observations with structured features, enabling fine-grained control, while a high-frequency action module predicts future tactile states and actions, adapting dynamically to changes during execution. Experimental results show that ReTouch significantly outperforms existing baselines, achieving an 18.4 to 23.8 percentage point improvement in success rates across various manipulation tasks.
ReTouch achieves a remarkable 18.4% improvement in success rates for dexterous manipulation by continuously refining tactile predictions in real-time.
Fusing tactile signals has proven effective for contact-rich manipulation, enabling robots to perceive contact states and adapt to rapidly changing physical interactions. Yet effectively integrating tactile feedback into dexterous manipulation remains underexplored. In this work, we introduce ReTouch, a vision-language-action model (VLA) that supports contact-rich dexterous manipulation through tactile predictions continually refined online using execution-time feedback. ReTouch builds on two main innovations for tactile representation and closed-loop action generation. First, its Tactile-Patch Encoder represents tactile observations as structured tactile patch features that preserve finger identity and local contact structure, providing contact cues for fine-grained dexterous control. Second, its high-frequency action module jointly predicts future tactile states and action chunks and refines both using incoming tactile feedback during execution. This closed-loop refinement keeps tactile predictions aligned with evolving physical interactions, enabling responsive action correction and improving robustness to contact changes and execution errors. We further introduce XHT-Dataset, comprising 900 real-world demonstrations across seven contact-rich tasks collected on an XHand--UR7e platform, and evaluate ReTouch through closed-loop real-robot experiments. ReTouch surpasses the strongest baseline by 18.4 and 23.8 percentage points in average success rate under standard and challenging conditions, respectively, demonstrating its effectiveness and robustness.
