Search papers, labs, and topics across Lattice.
This study systematically evaluates the performance of Vision-Language-Action (VLA) models across ten languages using the translated LIBERO benchmark, revealing a significant 30-50% drop in success rates for non-English instructions. The analysis identifies that language sensitivity varies across task execution steps, with certain steps being highly dependent on language while others remain agnostic. To address these challenges, the authors propose a step-wise intervention that aligns representations based on language sensitivity, leading to improved performance in multilingual contexts.
Language sensitivity in VLA models is a step-wise control problem, with certain task steps causing up to 50% performance degradation under non-English instructions.
Vision-Language-Action (VLA) models have shown strong performance in language-conditioned robotic manipulation, yet their robustness to linguistic variation remains poorly understood. In this work, we present the first systematic multilingual evaluation of VLA models by translating the LIBERO benchmark into ten languages, revealing severe performance degradation under non-English instructions, with success rates dropping by 30-50%. Through fine-grained analysis of task executions, we find that language influence is highly non-uniform across steps: certain steps exhibit strong language dependence and dominate overall task failure, while others are largely language-agnostic. Based on this insight, we propose a step-wise inference-time intervention that aligns representations according to step language sensitivity, substantially improving performance under linguistic variation. Our results indicate that language robustness in VLA models is fundamentally a step-wise control problem, highlighting the importance of temporally structured analysis for reliable embodied agents.
