Track synthesis, materials and catalysis as it is published.
Search papers, labs, and topics across Lattice.
WorldPlay2 tackles the dual failure modes of real-time interactive world models—heterogeneous control ambiguity and explosive context during long-horizon distillation—by coupling a factorized control interface with compressed memory distillation. The architecture disentangles frame-level action signals from structured semantic attributes (scene, identity, events) while sharing compact memory tokens between an autoregressive student and a bidirectional teacher for efficient clip-wise score evaluation. Stabilized by a few-step initialization and rollout replay mechanism dubbed Stable Forcing, the model achieves state-of-the-art long-horizon visual consistency under real-time interactive latencies.
Decoupling structured semantics from frame-level actions and distilling autoregressive rollouts via shared, compressed memory tokens solves the long-standing trade-off between real-time responsiveness and multi-step drift in interactive world models.
Interactive world models require responding in real time to versatile controls and maintaining long-horizon consistency. However, modeling heterogeneous controls remains difficult, while explosive contexts and unstable distillation impede achieving both long-horizon consistency and real-time responsiveness. In this paper, we present WorldPlay2, an interactive world model that couples a factorized hybrid control interface with a co-design of compressed memory and stable distillation. 1) Our factorized hybrid control interface integrates frame-aligned action control with structured semantic control that explicitly disentangles scene appearance, character identity, and dynamic semantic events, thereby facilitating effective control learning. 2) To achieve efficient long-horizon modeling, we compress historical contexts into compact memory tokens shared by the autoregressive student and the bidirectional teacher. This design enables clip-wise, memory-conditioned score evaluation instead of jointly processing an entire long rollout, substantially reducing distillation overhead. 3) We further propose Stable Forcing, which initializes the autoregressive student via a few-step strategy and leverages full-rollout replay to preserve the quality of long-horizon rollouts, ensuring robust and stable distillation. Extensive experiments demonstrate the strong generalizability of our model and its superior performance compared to existing methods.