Search papers, labs, and topics across Lattice.
This paper introduces ViTaR, a novel approach that enhances Vision-Language-Action (VLA) models by integrating visuo-tactile feedback as an execution modulator rather than an action generator, thereby preserving the pretrained capabilities of the VLA. By decomposing adaptation into Effect-Guided Modeling and Residual Action Modulation, ViTaR effectively applies bounded corrections based on local contact events without risking catastrophic forgetting. The method demonstrates a significant performance boost, achieving a 61.3% average success rate on the UniVTAC benchmark, outperforming both its frozen VLA base and existing tactile baselines by a substantial margin.
Tactile feedback can enhance VLA models without compromising their pretrained knowledge, leading to a 30.6 percentage-point improvement in manipulation tasks.
As Vision-Language-Action (VLA) models scale toward real-world deployment, contact-rich manipulation exposes a critical blind spot: these policies encode broad visual-semantic priors yet remain unaware of local contact events, producing identical actions whether contact is established, lost, or destabilized. Existing remedies either modify VLA internals, risking catastrophic forgetting, or demand online reinforcement under near-failure contact conditions. Both grant tactile unbounded influence over action generation, conflicting with the priors that make VLAs generalizable. We introduce ViTaR, which reframes tactile feedback from an action-generating perceptual input to an execution modulator that selects and scales bounded residual corrections atop a frozen VLA, preserving pretrained capabilities by construction. ViTaR decomposes adaptation into two stages: Effect-Guided Modeling determines whether and which correction is locally justified via outcome-grounded preference evidence, and Residual Action Modulation converts this evidence into a residual choice with continuously scaled gain from real-time visuotactile observations. On the UniVTAC benchmark spanning seven contact-rich tasks, ViTaR achieves 61.3% average success, a 30.6 percentage-point improvement over its frozen VLA base that also surpasses purpose-built tactile baselines. Physical-robot experiments confirm that bounded tactile modulation transfers to real sensor noise and dynamics.
