Search papers, labs, and topics across Lattice.
This paper introduces a scalable framework for training visual world models to solve general tasks by integrating self-distillation with reinforcement learning. By leveraging a vision-language model to generate task solutions from unlabeled scene images, the approach enables the distillation of execution knowledge into a more efficient Executor model, which operates without the need for costly paired task-execution videos. Experimental results demonstrate that the Executor outperforms the original Demonstrator and effectively transfers to robotic tasks, showcasing significant advancements in task-solving capabilities of pretrained video generators.
Self-distillation enables visual world models to solve tasks without costly video supervision, achieving superior performance in both simulated and robotic environments.
Pretrained video generators are promising visual world models that exhibit emergent task-solving abilities; however, their reliance on detailed textual descriptions limits their direct use for planning and decision-making. Existing approaches either outsource this reasoning to language or vision-language models, or rely on supervised fine-tuning with paired task-execution videos, which are costly to collect and difficult to scale. We propose a scalable framework that elicits task-solving ability in such models by combining self-distillation with reinforcement learning. Given an unlabeled scene image, a vision-language model generates a candidate task and a detailed step-by-step solution. The solution conditions a pretrained video diffusion model, the Demonstrator; we distill its behavior into an Executor conditioned only on the image and a short task prompt. This transfers execution knowledge from caption-guided generation to instruction-conditioned task solving without curated task-video supervision. We further improve the Executor with reinforcement learning from VLM feedback, exploiting the asymmetry between judging whether a sampled video satisfies a task and generating the solution. Experiments on our proposed WorldTasks-Benchmark and the DreamGen robotics benchmark show that the Executor surpasses the Demonstrator under our VLM-based evaluation protocol and transfers competitively to robotic tasks.
