Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper investigates the trade-off between explicit future video generation and fast latent action execution in World Action Models (WAMs), evaluating generalization across environmental perturbations, data efficiency, and new tasks. The authors discover that latent WAMs degrade significantly outside of in-distribution settings, but that the generalization gains of explicit WAMs stem almost entirely from the initial step of future token preparation rather than full video synthesis. Based on this, they introduce Simple-WAM, which executes only a single forward pass over fully noised video tokens with an adapted training noise schedule, matching explicit WAM generalization at latent WAM inference speeds in simulation and real-world tasks.
World action models do not need to render clean future video to generalize: conditioning on a single forward pass of pure noise captures virtually all the robustness benefits of multi-step diffusion at a fraction of the compute.
World action models (WAMs) predict the future alongside actions during \emph{training}. Due to the heavy computation cost of video denoising, whether the future must still be generated during \emph{inference} is disputed: Explicit WAMs denoise it into clean frames along with every action chunk, whereas Latent WAMs discard it entirely for acceleration. We find that latent WAMs, despite matching explicit ones on in-distribution tasks, fail to retain the generalization benefits that originally motivated WAMs. To demonstrate this, we evaluate generalization along three axes: \emph{environmental perturbation}, \emph{data efficiency}, and \emph{task generalization}. Controlled comparisons with a matched backbone, training data, and budget reveal consistent degradation across all three axes when the action expert no longer conditions on future representations. Further analysis shows that the gap arises almost entirely from the first denoising step: the benefit comes from \emph{preparing} the future, not \emph{generating} it. We therefore propose \textbf{Simple-WAM}, which simplifies future modeling into a single forward pass of fully noised video tokens and adapts the training-time noise schedule to this inference behavior. Across simulation and real-world tasks, Simple-WAM achieves the best of both worlds, leading explicit WAMs in generalization performance with efficiency comparable to Latent WAMs. Project Page: \href{https://zrporz.github.io/Simple-WAM-Web/}{\textcolor{panton}{\texttt{https://zrporz.github.io/Simple-WAM-Web}}}
Tsinghua AI