Search papers, labs, and topics across Lattice.
This paper introduces a two-stage training framework for Vision-Language-Action (VLA) models that incorporates pretraining of the action module with motion priors to enhance cross-embodiment robot manipulation. By first employing a flow-matching-based encoder-decoder to learn temporal motion structures from unconditioned action trajectories, the authors effectively establish a robust action prior before aligning visual and language features in the second stage. Experimental results demonstrate that this approach significantly improves convergence speed, success rates, and overall performance in both simulated and real-world tasks compared to traditional VLA training methods lacking action priors.
Pretraining action modules with motion priors can drastically enhance VLA model performance, achieving faster convergence and better success rates in complex robot manipulation tasks.
Most Vision-Language-Action (VLA) models build on a Vision-Language Model (VLM) backbone by attaching an action module and optimizing the full policy jointly. This design inherits strong visual and linguistic priors from the VLM, but leaves the action module to learn physical motion almost from scratch. As a result, the policy lacks an explicit motion prior, forcing early optimization to simultaneously discover temporal action dynamics and cross-modal alignment, a challenge further amplified in cross-embodiment settings. In this work, we propose to pretrain the action module with motion priors before cross-modal VLA alignment. Specifically, we introduce a two-stage training framework that equips the action module with cross-embodiment temporal motion structure before VLA training begins. In Stage~1, a lightweight flow-matching-based encoder-decoder action module efficiently learns temporal motion structure solely from unconditioned action trajectories, without processing visual or language tokens. In Stage~2, this learned prior is transferred to VLA training through decoder reuse and early-stage latent distillation, aligning visual-language features with the action embedding space while still allowing end-to-end policy refinement. In addition, the trained encoder serves as a compact history compressor, summarizing state-action histories into a single temporal context token for history-aware modeling at negligible cost. Extensive experiments across 13 diverse cross-embodiment tasks on both simulated and real-world platforms validate the effectiveness of our approach. Compared with VLA training without action priors, our model achieves faster convergence, higher success rates, and substantially stronger performance on data-scarce real-world tasks. Moreover, scaling up the action data in Stage~1 yields a more generalizable action prior that directly improves downstream VLA performance.
