Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper advances tactile-reactive manipulation in robotics by introducing a large-scale dataset and a novel Mixture-of-Transformers (MoT) architecture that effectively integrates high-frequency tactile signals. The authors address key limitations in current Vision-Language-Action (VLA) models, such as static encoders and a lack of diverse training data, enabling more agile and dexterous robotic manipulation. The results show a significant improvement, with the proposed method achieving over 30% higher average success rates across 12 complex manipulation tasks compared to the strongest baseline.
Tactile-reactive policies can boost robotic manipulation success rates by over 30% through innovative data collection and a new Mixture-of-Transformers architecture.
The ability to react dynamically to tactile signals has long been considered crucial to agile human-level dexterity. Yet contemporary learning-based Vision-Language-Action (VLA) models for robotic manipulation generally either overlook the tactile modality or are limited to encoders with static cues, due in part to the scarcity of diverse training data and standardized evaluation, architectural constraints in current VLA models, and limitations of static tactile encoders. In this paper, we push the frontier of tactile-reactive manipulation by addressing all of these limitations. We propose a large-scale, 100-hour tactile-rich dataset collected via a novel, data-efficient recipe that prioritizes elementary motor primitives. To effectively exploit naturally high-frequency touch signals without sacrificing the existing capabilities of existing VLAs, we introduce a variable-rate Mixture-of-Transformers (MoT) architecture equipped with a novel temporal tactile VQ-VAE encoder. We demonstrate the effectiveness of tactile-reactive policies on 12 manipulation tasks requiring delicate force control and deformable object manipulation, achieving over 30% higher average success rate than the strongest baseline.
BAIR
NVIDIA