World Models & Planning - Weekly Roundup

LLMs are revolutionizing conversational AI research, and this survey offers a structured guide to navigating the rapidly evolving landscape of LLM-powered user simulation.

Text-to-video models can now learn geometrically consistent world dynamics via reinforcement learning, without expensive architectural changes.

AI can now pilot jet trainers to avoid ground collisions, even with limited visibility.

Discrete diffusion policies, typically used for image generation, turn out to be surprisingly effective and efficient asynchronous executors for robots acting in dynamic environments, outperforming traditional continuous control methods.

Zero-shot Sim2Real transfer for a humanoid ballbot is now possible thanks to a friction-aware RL framework and high-fidelity simulation that models omni-wheel mechanics.

LLMs are revolutionizing conversational AI research, and this survey offers a structured guide to navigating the rapidly evolving landscape of LLM-powered user simulation.

Text-to-video models can now learn geometrically consistent world dynamics via reinforcement learning, without expensive architectural changes.

AI can now pilot jet trainers to avoid ground collisions, even with limited visibility.

Discrete diffusion policies, typically used for image generation, turn out to be surprisingly effective and efficient asynchronous executors for robots acting in dynamic environments, outperforming traditional continuous control methods.

Zero-shot Sim2Real transfer for a humanoid ballbot is now possible thanks to a friction-aware RL framework and high-fidelity simulation that models omni-wheel mechanics.

Successfully backing up a trailer without jackknifing or hitting anything just got easier thanks to a new path-planning algorithm that respects the physics of articulated vehicles.

Separating geometry from logic with fuzzy path constraints yields motion planning specifications that are both more intuitive for humans and more amenable to learning from demonstrations.

Autonomous vehicles can drive more efficiently by using a new metric that links real-time acceleration decisions to overall travel time.

Multi-robot motion planning can be accelerated by over 850X, enabling solutions in milliseconds, by exploiting SIMD parallelism with vector-accelerated primitives.

Ditch expensive robot trials: a novel "betting" framework lets you accurately predict real-world robot performance using only cheap simulations.

AtomWorld achieves the previously impossible: simulating the degradation of reactor pressure vessel steel at the atomistic level across year-and-meter scales.

Achieve real-time learning-based control of complex robotic systems by exploiting differential flatness for dramatic speedups in MPC computation.

Forget slow, expensive real-world trials: dWorldEval's discrete diffusion world model lets you evaluate robot policies across thousands of environments and tasks with unprecedented speed and accuracy.

The fragmented field of world modeling can now be unified under a "levels x laws" taxonomy, revealing critical gaps in autonomous model revision and decision-centric evaluation.

Stop guessing which interactive video model is best: WorldMark offers the first apples-to-apples comparison across leading models on identical scenes and trajectories.

Controller design can be effectively framed as inference, enabling efficient trajectory and policy optimization via tempered sampling.

Integrating deep learning forecasting with MILP optimization slashes inventory costs by 5.4% and stockouts by 27.5% in textile and PPE supply chains.

RL policies don't have to be temporally incoherent messes: shaping action probabilities with dynamical priors unlocks structured, interpretable decision-making.

LLMs can plan complex trips far more effectively when their reasoning is structured as a "forest" of parallel behavior trees, each handling a subtask and coordinated globally.

Forget complex architectures: the secret to self-improving LLM agents lies in teaching them how to *interpret* their past failures, not just remember them.

Autonomous vehicles can now plan trajectories 10x faster without sacrificing performance, thanks to a novel architecture that learns complex driving behaviors in latent space during training.

DINOv3 representations and diffusion-based planning enable a visual tracker that's both robust to occlusions and discriminative enough to avoid visually similar distractors.

Optimality guarantees are now possible when jointly optimizing robot design, fleet composition, and task planning for heterogeneous multi-robot systems.

Achieve robust robot manipulation across diverse viewpoints without camera calibration by synthesizing novel views with a geometry-aware video diffusion model.

Forget expensive real-world robot training: Hi-WM lets humans directly edit a robot's simulated reality, turning world models into powerful, reusable playgrounds for failure recovery.

Optimal robot exploration can be achieved by framing SLAM as a POMDP with a geometry-aware exploration cost, enabling near-optimal policy learning.

Humanoid robots can now adapt to diverse environments without task-specific tuning by selectively "relaxing" joints, mimicking how humans exploit weightlessness for stability.

Humanoid robots can now seamlessly transition between fighting skills thanks to a novel policy gating approach that ensures stability and smoothness.

Fine-tuning VLMs with action-aligned language supervision and terrain-aware preference optimization unlocks more robust off-road autonomous driving, outperforming prior approaches on key traversability metrics.

LLMs can write better stories if they plan the plot on a graph first.

Uncover more LLM agent failures, faster: DIVERT's diversity-guided user simulation finds more bugs per token than standard rollout methods.

Image editing models can learn to solve visual planning puzzles with finetuning, but still lag far behind humans in zero-shot efficiency, revealing a key gap in neural visual reasoning.

World-model-based planning enables reliable robotic manipulation in complex industrial settings where reactive policies crumble.

Stop guessing and start knowing: this framework accurately predicts *your* EV's energy consumption by learning your driving style and integrating it with detailed map data.

Get 82x faster Bayesian inference for equipment monitoring by replacing MCMC with neural nets trained on simulated data.

Extracting temporal geometry from generative models can boost reinforcement learning performance by over 2x without changing the optimal policy.

Sampling plausible configurations of digital twins can reveal multiple valid parameterizations, enhancing model adaptation in complex natural systems.

GNNs can predict network traffic flow with surprising accuracy, particularly in pinpointing connection endpoints.

Combining heuristics with learned models for graph sparsification yields significantly sparser and more reliable candidate graphs for TSP solvers, outperforming purely heuristic or learned approaches, especially as problem size increases.

World models can navigate blood vessels autonomously with higher success rates than standard RL, paving the way for safer robotic stroke treatments.

Ontology augmentation transforms LLMs into robust reasoning agents, significantly boosting performance in complex planning tasks.

Forget meticulously annotating subtasks – SuperIgor lets language models self-learn to generate and refine instruction-following plans through RL feedback.

Solving NP-hard combinatorial optimization problems like QAP just got a whole lot faster, thanks to a novel MCMC finetuning approach that achieves near-zero optimality gaps.

Stop passively waiting for retrieval cues – ProactAgent proactively asks for information from its memory and skills, leading to significant gains in lifelong learning performance.

Visualize realistic flood scenarios in any location using only 360° video and readily available 2D building footprint data.

Generating realistic and stable human co-manipulation motions is now possible by explicitly modeling object affordances and spatial configurations within a flow-matching framework.

Achieve 2.6x faster autoregressive world model inference without retraining by caching and selectively reusing block-level residuals across generation chunks.

Robots can now map hazardous scalar fields more safely and efficiently by combining Gaussian Process regression with Hough Transform-based identification of unsafe regions.

Achieve superhuman dexterity: ALAS unlocks robust long-horizon task completion by decoupling environment understanding from motor control, enabling generalization across diverse human-scene interaction scenarios.

Forget conservative approximations – this work delivers provably tighter safety guarantees for robots navigating dynamic, uncertain environments.

Train robots to grasp blindly with tactile feedback at 800+ FPS using this new, fast, and accurate soft-body simulator.

Prioritizing safety specs in robot motion planning doesn't have to be a computational nightmare: this work shows how to do it efficiently by cleverly reformulating the problem.

Cooperative driving in mixed traffic gets a boost: adaptively learning human driver preferences within a potential game framework significantly improves safety and efficiency in real-world field tests.

LLMs can learn to play complex games far more effectively by co-evolving a skill bank with a decision-making agent, enabling consistent long-horizon decision-making.

Forget static scenes: Paparazzo lets robots actively map *moving* 3D objects with unprecedented accuracy.

Stop guessing and start planning: a new data-driven framework slashes vaccine stockouts while optimizing inventory, even with unpredictable demand and strict shelf-life constraints.

Force-guided robot finesse is now within reach: MATCH learns hybrid position-force control that doubles success rates in real-world, high-noise insertion tasks compared to traditional pose control.

Entropy regularization makes planning provably easy: SmoothCruiser achieves polynomial sample complexity in MDPs where standard methods fail.

Automating particle accelerator beamline optimization with reinforcement learning now rivals traditional methods, opening the door to faster and more efficient beam tuning.

Escaping the tyranny of Bellman's curse, a new method leverages multi-step transitions to achieve higher-order accuracy in continuous-time policy evaluation, outperforming traditional one-step recursion.

Safety guarantees for black-box control systems can be derived from the geometry of learned feasibility constraints, not just unknown dynamics.

Agents can now explore environments more efficiently by thinking like humans, prioritizing key landmarks and semantic information during online memory construction.

Simply feeding more history to visuomotor policies hurts performance; GMP solves this by learning when and what to remember, boosting success rates by 30% on memory-intensive robotic tasks.

Forget static simulations – YAIFS lets you build interactive, agent-driven cloud-edge environments controllable via LLMs and multi-agent systems.

Diffusion models can now handle arbitrary, unordered sparse inputs for 3D reconstruction, achieving robust and scalable performance across irregular viewpoints and long trajectories.

Achieve model-free RL sample efficiency with model-based control performance by sidestepping multi-step rollout errors via one-step Koopman dynamics predictions.

Semantic masks are all you need: predicting mask dynamics in world models yields surprisingly robust and generalizable robot policies compared to predicting raw pixels.

Human-robot teams get a fluency boost: RAPIDDS learns your quirks and preferences over time, adapting both task schedules and robot motions for smoother, more efficient collaboration.

Autonomous robots can now explore and map environments more effectively by using Gaussian processes to plan paths that adapt to multi-modal sensor data and uncertainty.

Humanoid robots can master diverse gaits like walking, running, and stair climbing with a single policy, but only if you selectively apply motion priors to stabilize specific skills.

Today's visually impressive video world models still fail to produce physically plausible robot actions, revealing a critical gap between visual realism and embodiment.

Sobolev training of diffusion policies slashes trajectory optimization time by up to 20x by preventing compounding errors, even when learning from limited data.

Contact-aware reconstruction transforms how we achieve realistic human-scene interactions in 3D environments, correcting artifacts that have plagued previous methods.

Generate navigable, 3D-consistent simulations of real-world locations with arbitrary weather and dynamic object configurations using only geo-registered video data.

UniT reveals that a unified physical language can enable humanoid robots to leverage vast human data for superior learning and adaptability.

Forget reward engineering: this work shows LLMs can self-evolve and outperform larger models by learning to explore and summarize new environments autonomously.

STRATAGEM reveals that selectively reinforcing reasoning trajectories can dramatically enhance a model's ability to transfer reasoning skills across diverse tasks, especially in complex mathematical scenarios.

Transforming sparse driving log observations into complete 3D assets could revolutionize simulation fidelity in autonomous vehicle development.

Aether automates network change validation with 100% error detection and rapid processing, transforming a traditionally manual and error-prone task into a streamlined workflow.

Modeling human actions as executable programs not only improves action recognition, but also unlocks more data-efficient and interpretable models compared to standard monolithic approaches.

Stop mis-delegating: Context-aware delegation, informed by uncertainty, boosts multi-agent performance by up to 9% on complex benchmarks.

Accurately simulating multi-agent interactions with consistent multi-view video is now possible thanks to MultiWorld, a framework that scales to many agents and viewpoints.

Agent-World reveals that self-evolving environments can dramatically boost agent performance, outperforming established models by leveraging dynamic task synthesis.

Platypoos adapts seamlessly to unknown reward scales, achieving optimal sample complexity in planning under uncertainty.

Prioritizing bridge maintenance for disaster resilience gets a data-driven upgrade: a new model classifies bridges by their critical roles in supply chains, medical access, and residential protection using open data and graph neural networks.

Forget trajectory forecasting – TacticGen generates *adaptable* football tactics, bridging the gap between predicting what *will* happen and prescribing what *should* happen to win.

Drifting Models get a friction boost: DMF matches or beats Optimal Flow Matching on FFHQ image translation while slashing training compute by 16x.

Neural fields can seamlessly integrate into environmental modeling workflows, providing stable and scalable representations from sparse ecological data.

Sonata outperforms traditional models in clinical kinematic assessments, achieving better fall-risk predictions with a fraction of the parameters.

WorldDB's unique graph-of-worlds architecture allows for unprecedented accuracy in long-term memory tasks, outperforming existing systems by a notable margin.

Forget expensive, error-prone math problems: PDDL planning offers a surprisingly effective and scalable route to training better Process Reward Models for LLM reasoning.

Training tool-calling agents with just an 8B language model outperforms traditional methods that depend on expensive resources, reshaping the landscape of tool learning.

Unlock scalable cooperative humanoid manipulation by transferring skills from readily available single-agent data to multi-agent scenarios.

Hierarchical planning with vision-language models and decoupled arm-hand control unlocks dexterous grasping in cluttered, tiered workspaces where traditional methods falter.

Forget brittle physics models: DART uses online learning to give dual-armed robots waiter-like dexterity in balancing objects on a tray.

Uncover hidden vulnerabilities in your Byzantine Fault Tolerant systems with a Digital Twin that exposes timing-sensitive exploits and adversarial delays.

SVGD lets autonomous vehicle testers find 28% more safety violations by intelligently exploring the simulation space, even with complex systems like Apollo and Autoware.

AlphaEarth embeddings live on a surprisingly twisted, non-Euclidean manifold, and understanding this geometry unlocks better environmental reasoning.

Flow-based offline RL gets a geometric upgrade: Fisher Decorator uses a local transport map to ditch isotropic regularization and unlock state-of-the-art performance.

Guaranteeing stability for complex robot locomotion just got easier: HALO learns low-dimensional models that accurately predict stability regions in the full state space.