Search papers, labs, and topics across Lattice.
The paper introduces Chain-of-World VLA (CoWVLA), a new Vision-Language-Action (VLA) model paradigm that combines world-model temporal reasoning with a disentangled latent motion representation. CoWVLA uses a pretrained video VAE to extract structure and motion latents from video segments, and then learns to infer a continuous latent motion chain from an instruction and initial frame to predict the segment's terminal frame. Experiments on robotic simulation benchmarks demonstrate that CoWVLA outperforms existing world-model and latent-action approaches, suggesting it's a more effective VLA pretraining paradigm.
By disentangling structure and motion in the latent space, CoWVLA achieves superior visuomotor learning compared to standard world-model and latent-action approaches.
Vision-Language-Action (VLA) models are a promising path toward embodied intelligence, yet they often overlook the predictive and temporal-causal structure underlying visual dynamics. World-model VLAs address this by predicting future frames, but waste capacity reconstructing redundant backgrounds. Latent-action VLAs encode frame-to-frame transitions compactly, but lack temporally continuous dynamic modeling and world knowledge. To overcome these limitations, we introduce CoWVLA (Chain-of-World VLA), a new"Chain of World"paradigm that unifies world-model temporal reasoning with a disentangled latent motion representation. First, a pretrained video VAE serves as a latent motion extractor, explicitly factorizing video segments into structure and motion latents. Then, during pre-training, the VLA learns from an instruction and an initial frame to infer a continuous latent motion chain and predict the segment's terminal frame. Finally, during co-fine-tuning, this latent dynamic is aligned with discrete action prediction by jointly modeling sparse keyframes and action sequences in a unified autoregressive decoder. This design preserves the world-model benefits of temporal reasoning and world knowledge while retaining the compactness and interpretability of latent actions, enabling efficient visuomotor learning. Extensive experiments on robotic simulation benchmarks show that CoWVLA outperforms existing world-model and latent-action approaches and achieves moderate computational efficiency, highlighting its potential as a more effective VLA pretraining paradigm. The project website can be found at https://fx-hit.github.io/cowvla-io.
Tsinghua AI