Search papers, labs, and topics across Lattice.
The paper introduces MultiGen, a diffusion game engine that incorporates an explicit external memory to enable user-editable and multiplayer interactive environments. By decoupling generation into Memory, Observation, and Dynamics modules, MultiGen allows users to directly manipulate the environment's structure through an editable memory representation. This architecture facilitates real-time multiplayer rollouts with coherent viewpoints and consistent cross-player interactions, overcoming limitations of existing video world models.
Imagine Minecraft, but you can edit the world's code in real-time with friends, thanks to a diffusion model that treats the environment as editable external memory.
Video world models have shown immense promise for interactive simulation and entertainment, but current systems still struggle with two important aspects of interactivity: user control over the environment for reproducible, editable experiences, and shared inference where players hold influence over a common world. To address these limitations, we introduce an explicit external memory into the system, a persistent state operating independent of the model's context window, that is continually updated by user actions and queried throughout the generation roll-out. Unlike conventional diffusion game engines that operate as next-frame predictors, our approach decomposes generation into Memory, Observation, and Dynamics modules. This design gives users direct, editable control over environment structure via an editable memory representation, and it naturally extends to real-time multiplayer rollouts with coherent viewpoints and consistent cross-player interactions.
Stanford HAI