Search papers, labs, and topics across Lattice.
This paper introduces WorldWeaver (W^2), a novel streaming multi-agent video diffusion model that incorporates cross-agent world state registers to maintain shared world information across agents and views. By employing learnable tokens that dynamically update after each generated chunk, W^2 enhances the logical consistency and quality of generated video sequences in multi-agent environments. Experimental results in two-agent Minecraft scenarios demonstrate significant improvements in both generation quality and coherence compared to existing autoregressive models.
Explicitly modeling shared world states in multi-agent settings can drastically enhance video generation quality and logical consistency.
Multi-agent interactive world models should not only generate consistent observations, but also maintain world states that persist across agents and evolve across views. Existing autoregressive video diffusion pipelines carry forward observation history as conditioning context, which makes shared state difficult to maintain in multi-agent and multi-view settings. We present WorldWeaver (W^2), a streaming multi-agent video diffusion model that augments rollout with cross-agent world state registers: learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk. We ground these registers with supervision signals spanning individual agent status, global state views including bird's-eye views, and scene text. We further improve the architecture with a Mixture-of-Transformers design that uses separate weights for world state modeling and visual frame modeling. Extensive experiments in two-agent Minecraft video generation show that explicit world-state modeling improves logical consistency and generation quality.
