Search papers, labs, and topics across Lattice.
This paper introduces LiLa-WAM, a lightweight latent world-action model designed for robotic manipulation that effectively predicts future states and generates actions within a compact latent space. By utilizing a Visual Transition Token for task specification, the model circumvents the computational overhead typical of existing world-action models, enabling efficient end-to-end training on a single 24GB GPU. Experimental results show LiLa-WAM achieving a 90.48% success rate across 50 tasks in the RoboTwin environment, highlighting its practical applicability in real-world robotic scenarios.
Achieving over 90% success in robotic tasks with a model that can be trained on a single GPU challenges the notion that high performance requires massive computational resources.
World-action modeling has emerged as a promising paradigm for robotic control, as it empowers models to go beyond reacting to observations and anticipate how a scene will evolve. However, existing WAMs often incur substantial computational overhead. Pixel-space methods often allocate substantial capacity to visual details that may not be directly relevant to control, while some latent-space methods require multi-stage training to construct the reasoning space. The resulting training cost can make such methods difficult to train under modest computational budgets. In this work, we propose LiLa-WAM, a lightweight world-action model that reasons about the future in a compact latent space and can be trained end-to-end on a single 24GB GPU. Its core design is a compact latent reasoning space jointly shaped by future-state prediction and action generation, which keeps the model lightweight while remaining well aligned with control. For task specification, we further propose the Visual Transition Token(VTT), a language-free task representation that encodes each task as a direction in visual feature space. Experiments on RoboTwin~2.0, LIBERO, and real-robot tasks demonstrate LiLa-WAM's effectiveness, achieving 90.48\% success across 50 RoboTwin tasks with single-GPU training.
