Search papers, labs, and topics across Lattice.
This paper introduces LaWAM, a Latent World Action Model that enhances robot control by conditioning action policies on compact latent visual subgoals rather than computationally intensive video generation. By leveraging a latent-action-conditioned Latent World Model trained in the latent space of a vision foundation model, LaWAM efficiently predicts future observation features to inform scene evolution. The approach achieves state-of-the-art success rates in various benchmarks while significantly reducing inference latency, demonstrating a practical advancement in dynamics-aware robot policies.
LaWAM achieves up to 24x lower latency than traditional pixel-space models while maintaining state-of-the-art performance in robot control tasks.
Vision-Language-Action models (VLAs) leverage large-scale vision-language pretraining for semantic robot control, but often lack explicit foresight into how robot actions change the scene. World-Action Models (WAMs) address this limitation by conditioning policies on predicted futures, yet existing approaches typically rely on computationally expensive video generation with substantial pixel-level redundancy. We present LaWAM, a Latent World Action Model that exposes predictive dynamics to robot policies through compact latent visual subgoals instead of reconstructed future video. At the core of LaWAM is a latent-action-conditioned Latent World Model (LaWM). We obtain LaWM by training a latent action model in the latent space of a pretrained vision foundation model and repurposing its forward decoder to predict future observation features for scene evolution. LaWAM then conditions action generation on these predicted latent visual subgoals to enable dynamics-aware robot control. LaWAM achieves state-of-the-art or competitive success rates (SRs) across LIBERO (98.6% SR), RoboTwin (91.22% SR), and real-world manipulation tasks while retaining low-latency inference. LaWAM runs in 187 ms per action-chunk prediction and achieves up to 24x lower wall-clock latency than pixel-space WAMs.
