Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
PLaW-VLA addresses long-horizon VLA control by learning task-relevant future state predictions in a pretrained representation space, avoiding costly low-level visual reconstruction. Its Mixture-of-Transformers architecture conditions actions on historical observations, task semantics, and these latent future states via causal attention. This approach yields significant performance gains on challenging benchmarks and achieves substantially lower inference latency compared to generative world-action models.
Predicting task-relevant latent futures, not raw pixels, unlocks efficient and effective long-horizon VLA control, outperforming reactive policies by over 11 percentage points.
Learning to predict how the world evolves can provide vision-language-action (VLA) policies with predictive context for long-horizon control, but its effectiveness depends on what future representation is modeled and how it conditions action generation. We introduce PLaW-VLA, which models task-relevant future states in a pretrained prediction-oriented representation space, reducing the need to predict control-irrelevant visual details. Built on a Mixture-of-Transformers architecture, PLaW-VLA conditions action generation on observation history, current task semantics, and predicted future states through structured causal attention. Experiments show a +11.8 percentage-point (pp) gain over reactive policies on RoboTwin Hard Horizon III and a +1.77 pp gain over reconstruction-oriented latent prediction on zero-shot LIBERO-Plus, supporting improved long-horizon control and generalization under distribution shift, respectively. By avoiding low-level visual reconstruction, PLaW-VLA lowers the burden of future prediction, enabling a lightweight latent world model with parallel future prediction and about 1/19 the inference latency of generative world-action modeling at comparable policy performance.
Amazon Science