Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper formalizes text-based world modeling as a steerable transition-dynamics problem, addressing the limitations of autoregressive (AR) models in capturing interdependent state anchors. By leveraging masked diffusion language models (MDLMs) and a curated dataset of over 239,000 grounded state-action trajectories, the authors demonstrate that MDLMs outperform AR models in coherence, groundedness, and rollout diversity, achieving significant performance improvements without environment-specific fine-tuning. The introduction of a plug-and-play GRPO training framework further enhances the applicability of their approach across diverse environments, yielding up to 47% absolute gains in zero-shot transfer scenarios.
Masked diffusion language models can outperform larger autoregressive models in text-based world modeling, achieving superior coherence and diversity without fine-tuning.
Recent growth in reinforcement learning (RL) has surfaced a need for diverse, specialized training environments. Hand-curated environments with fixed task and reward difficulties become ineffective signals as model performance improves, and sparse rewards over long horizons induce mode collapse on specific workflows or tool structures. World models that simulate environment states have matched pure rollout performance, making them promising for scaling diversity on-demand. However, autoregressive (AR) world models suffer from a left-to-right bias preventing conditioning on globally interdependent state anchors such as tool schemas, prior turns, and expected outcomes. We (i) formalize text-based world modeling as a steerable transition-dynamics problem decomposed into initial state, task context, tool schemas, domain rules, and steering directives, and (ii) curate 239,403 grounded state-action trajectories spanning nine open-source environments and twelve frontier model families. We compare AR LMs and masked diffusion language models (MDLMs), showing MDLMs, via bidirectional anchor-aware denoising, achieve better coherence, groundedness, and empirically validated rollout diversity than LLMs over 4x their parameter size, at comparable inference latency. We introduce a plug-and-play GRPO training framework with deterministic state checks, and perform zero-shot transfer ablations on three OOD environments (ScienceWorld, ALFWorld, AppWorld) across three 1.2B-7B agent backbones (LFM2.5, Qwen3, Mistral), achieving up to 47% absolute gains over baselines without environment-specific fine-tuning. We further conduct behavioral analysis of failure modes under adversarial scenarios and human evaluation on realism, outcome correctness, and training utility. We open-source our work to encourage research in this direction.