Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces LAWA, a novel architecture for World Action Models (WAMs) that utilizes compact latent actions to represent future intentions, significantly enhancing efficiency in generating future observations during robot control tasks. By employing a discrete tokenizer and action-free pre-training, LAWA achieves state-of-the-art success rates on RoboCasa, outperforming the Fast-WAM baseline while reducing inference latency by 42.9%. The results indicate that efficient future imagination can be retained without sacrificing performance, thus addressing challenges in generalization and latency in robotic applications.
Retaining future imagination through compact latent actions allows LAWA to outperform existing models while slashing inference latency by nearly 43%.
World action models (WAMs) improve robot control by modeling how observations evolve, but generating future observations at test time incurs substantial latency. Fast-WAM removes this process for efficiency; however, our matched implementations show lower generalization for Fast-WAM than for future-aware alternatives, especially with scarce robot demonstrations and in out-of-distribution scenarios. To bridge this gap, we introduce **LAWA**, a WAM architecture that uses compact latent actions as an operational representation of future intentions, enabling efficient test-time future imagination without generating future observations. Specifically, a discrete tokenizer enhanced by action-free pre-training produces manipulation-centric codebook targets. LAWA jointly denoises a continuous latent state anchored to these targets with executable action chunks while omitting the future-video branch at inference. On RoboCasa, LAWA achieves state-of-the-art average success rates of 65.6% and 80.8% in the few-shot and full data settings, improving over the matched Fast-WAM baseline by 9.6 and 4.5 points, respectively. It also preserves the performance level of the matched Joint-WAM variant while requiring 42.9% lower inference latency. LAWA also demonstrates competitive zero-shot robustness on LIBERO-Plus and superior performance on real-world tasks. These results show that future imagination need not be discarded: retaining it with compact latent actions yields an effective trade-off among performance, generalization, and latency. Code and models will be released.
Tsinghua AI