Search papers, labs, and topics across Lattice.
WALL-SS introduces a novel world model that leverages Scale-wise autoregressive Scaling to generate action-controllable, long-horizon visual simulations for robotic applications. By representing embodied trajectories as causal sequences of observations and actions, it enables flexible interaction and direct optimization through sequence probabilities, addressing the limitations of traditional predictive models. Experimental results demonstrate that WALL-SS significantly enhances action following, trajectory accuracy, and long-horizon consistency, while efficiently managing memory through scale-compressed representations.
WALL-SS achieves unprecedented long-horizon visual simulation for robots, improving action following and trajectory accuracy while maintaining coherence over extended interactions.
Generative world models provide robots with predictive models of how the world evolves under interaction, with growing potential for simulation, planning, policy evaluation, and robot learning. Beyond clip-level future prediction, a unified generative formulation should relate actions to consequences, support flexible horizons and continuous interaction, and enable reward-driven optimization. We introduce WALL-SS, a world model that generates visual futures through Scale-wise autoregressive Scaling, enabling action-controllable and long-horizon robotic simulation. WALL-SS represents embodied trajectories as causal sequences of temporally interleaved observations and actions, making action-dependent state transitions explicit while naturally supporting variable-length generation, streaming extension through reusable causal states, and direct optimization through sequence probabilities. To make this formulation effective over long horizons, we generate each future observation in a coarse-to-fine manner and develop three complementary components within the same hierarchy. Action-conditioned next-scale prediction injects scale-aligned action representations to improve action-future coupling and model both successful and failed behaviors. Scale-compressed long-horizon memory retains recent interactions at fine resolution while compressing distant observations and actions, with scale-wise dream forcing enhancing robustness to self-generated context. Finally, on-policy alignment optimizes autoregressive visual dynamics with action-following and long-term consistency rewards while preserving the pretrained visual distribution. Experiments show that WALL-SS improves action following and trajectory accuracy, supports coherent minute-long streaming rollout under bounded memory, and consistently benefits from on-policy alignment in reducing action drift and long-horizon inconsistency.
