Search papers, labs, and topics across Lattice.
The authors introduce LA4VLA, a novel language-action pretraining framework designed to enhance Vision-Language-Action (VLA) models by enabling them to learn action priors without relying on visual observations. By decomposing expert demonstration trajectories into atomic action segments and pairing them with low-level action descriptions, they create the LA4-33K dataset, which consists of 33,000 language-action episodes. Experimental results demonstrate that policies pretrained with LA4VLA significantly outperform those trained with traditional VLA methods, achieving up to 45% higher success rates in real-world tasks when using a mixed pretraining approach.
Language-action pretraining can lead to VLA policies that are not only more robust but also less dependent on visual cues, achieving up to 45% higher success rates in real-world tasks.
Vision-Language-Action (VLA) models are commonly pretrained on robot demonstrations by jointly mapping visual observations and language instructions to actions. However, dense visual-action supervision can dominate the comparatively sparse language-action signal. As a result, policies may rely on visual shortcuts rather than learn how language conditions action execution, making them sensitive to visual variations. To address this limitation, we propose LA4VLA, a language-action pretraining framework that enables policies to acquire language-conditioned action priors without visual observations. These priors capture reusable manipulation skills shared across tasks and scenes, reducing reliance on scene-specific visual cues. Specifically, LA4VLA decomposes expert demonstration trajectories into atomic action segments and pairs each segment with a corresponding low-level action description. This yields LA4-33K, a dataset of 33K Language-Action (LA) episodes derived entirely from existing demonstrations without additional robot data collection. We further develop LA4VLA-1B, a lightweight 1B-parameter VLA model, and investigate three paradigms for incorporating language-action supervision into VLA learning: LA-only pretraining, sequential LA-to-VLA pretraining, and mixed LA-VLA pretraining. Across simulation and real-world tasks, LA-pretrained policies consistently outperform matched VLA-pretrained counterparts, while combining LA and VLA supervision leads to further gains. In particular, mixed LA-VLA pretraining improves the average success rate of LA4VLA-1B over the no-pretraining baseline by up to 17.8 and 45.0 percentage points in simulation and real-world tasks, respectively. These results establish LA4VLA as an effective and complementary pretraining strategy for building stronger and more robust VLA policies.
DAMO