Search papers, labs, and topics across Lattice.
This paper introduces WorldScape Policy 2.0, a novel World Action Model (WAM) that enhances robotic manipulation through reasoning-augmented long short-term memory, addressing limitations in temporal context and language supervision. By leveraging a causal short-term visual memory and a structured long short-term event memory, the model enables fine-grained multimodal controllability and autonomous planning from high-level instructions. Experiments reveal that WorldScape Policy 2.0 significantly outperforms existing methods in long-horizon planning and instruction adherence across both simulated and real-world environments.
WorldScape Policy 2.0 achieves unprecedented long-horizon autonomous planning by integrating reasoning-augmented memory with multimodal instruction processing.
World Action Models (WAMs) offer a promising paradigm for robotic manipulation by jointly modeling visual state transitions and robot actions. However, existing WAMs are constrained by limited temporal context, coarse episode-level language supervision, and predominantly text-only conditioning, which hinder task-progress tracking and fine-grained language-video-action grounding while limiting visual-context reasoning and cross-embodiment transfer. In this paper, we introduce WorldScape Policy 2.0, a controllable WAM with reasoning-augmented long short-term memory. Its causal short-term visual memory supplies recent observations as DiT prefill to preserve local interaction dynamics, while its long short-term event memory organizes historical VLM outputs into global-history, local-active, and event-boundary representations for progress-aware retrieval. The retrieved history augments perception and autoregressively generated planning tokens, yielding an implicit subgoal condition for autonomous planning; semantic forcing further transfers event-level instruction semantics into this latent planning pathway. To establish fine-grained multimodal controllability, we construct ManipEvent-5M, an event-grounded embodied pretraining dataset containing nearly 5 million event segments with aligned action trajectories, episode-level task instructions, segment-level subtask captions, goal images, and video demonstrations. These designs provide a unified interface for autonomous planning from high-level instructions and controllable execution from fine-grained text, goal-image, or video-context prompts. Experiments in both simulation and real-world platforms demonstrate superior capabilities in long-horizon autonomous planning, fine-grained instruction following and in-context adaptation.
