Search papers, labs, and topics across Lattice.
WALL-WM introduces a novel World Action Model that transitions from traditional chunk-centric optimization to an event-grounded approach in Vision-Language-Action (VLA) pretraining, leveraging semantically coherent action events as the fundamental learning unit. This shift addresses the granularity mismatch inherent in existing models, which struggle to align language, vision, and action within fixed-length prediction windows. Experimental results demonstrate that WALL-WM not only generalizes effectively across diverse behaviors and scenes but also achieves state-of-the-art performance in large-scale real-world evaluations.
By redefining action learning around semantic events, WALL-WM achieves unprecedented generalization across tasks and environments, outperforming traditional models.
WALL-WM is a World Action Model that shifts video-action learning from chunk-centric optimization to event-grounded Vision-Language-Action pretraining, using semantically coherent action events as the atomic unit of learning. Existing WAMs commonly initialize from multimodal or video foundation models and then optimize fixed-length action chunks conditioned directly on the current observation and instruction. Although convenient, this chunk-centric formulation creates a fundamental granularity mismatch. Language describes semantic goals and events, vision evolves through continuous scene dynamics, and actions operate at control-level timescales; forcing all three into the same fixed-length prediction window turns VLA training into short-horizon correlation fitting. WALL-WM addresses this mismatch by organizing both supervision and data around semantic events. Specifically, it pairs event-grounded VLA pretraining with a data ecosystem built from event-level captions and cluster-balanced sampling, enabling scalable learning over diverse behaviors, scenes, and task structures. From the same event-pretrained backbone, WALL-WM supports two complementary inference modes. The event mode consumes next-event descriptions and enables variable-length execution chunks, while the unified mode uses a VLM with Staircase Decoding to condition conventional fixed-length chunk inference while preserving a gradient-continuous VLA path. Together with Muon-optimizer-based large-scale pretraining infrastructure, WALL-WM provides a practical scale-up recipe for general-purpose WAMs. Experiments show that WALL-WM generalizes broadly across language, scenes, and tasks, achieving state-of-the-art performance in large-scale real-world generalization evaluation.
