Search papers, labs, and topics across Lattice.
This paper introduces MiniWorld, a framework designed for the efficient training of video world models from scratch, addressing the limitations of existing methods that rely on complex pipelines and pretrained models. By employing a block-causal Video Diffusion Transformer with Flow Matching in a latent space, MiniWorld achieves scalable and reproducible training while enhancing temporal modeling and stability through innovative noise scheduling and asynchronous denoising techniques. The results indicate that MiniWorld can be effectively trained on modest computational resources, making it accessible for broader research applications in embodied AI and interactive simulation.
MiniWorld enables the training of video world models from scratch in just a few days on a single 8-GPU server, democratizing access to advanced video modeling techniques.
Video world models predict future observations conditioned on historical observations and control signals, enabling long-horizon generation through autoregressive state transitions. Unlike conventional video generation models that primarily capture visual appearance and motion, video world models learn the underlying dynamics governing environment evolution under agent actions, providing a foundation for embodied AI and interactive simulation. Recent progress has largely relied on adapting pretrained video generation models through post-training or distillation. Although effective, these approaches often require complex training pipelines, substantial computational resources, and suffer from the mismatch between bidirectional pretraining and causal streaming inference. Recent studies have shown that training autoregressive video world models from scratch is feasible and scalable. However, the community still lacks a lightweight, transparent, and fully reproducible baseline trainable end-to-end with modest computational resources. We present MiniWorld, a reproducible framework for training streaming video world models from scratch. MiniWorld employs a block-causal Video Diffusion Transformer trained with Flow Matching in the latent space of a pretrained Video VAE. Building on Diffusion Forcing, it adopts a chunk-wise non-decreasing noise schedule and two-stage continued training to improve temporal modeling and stability. During inference, MiniWorld combines a rolling KV cache with pipelined asynchronous denoising for efficient streaming generation under bounded computation. The entire model can be trained within several days on a single 8-GPU server. By releasing the training and inference codebase and pretrained checkpoints, we hope MiniWorld will facilitate future research on video world modeling.
