World Models & Planning - Weekly Roundup

Forget grid layouts: Map2World lets you generate consistent 3D worlds from arbitrary segment maps, offering unprecedented control and scalability.

Simply detecting distribution shifts in visual MBRL is easy; the real challenge is applying the right action-level corrections, which this paper tackles with a novel local expert growth strategy.

Even the most advanced language models still lose money and demonstrate unsophisticated strategies when tasked with maximizing long-term bankroll growth in a realistic sports betting simulation, highlighting a significant gap in their sequential decision-making capabilities.

Semantic rollouts and town-adversarial regularization can significantly boost zero-shot driving performance in unseen CARLA towns, even without explicit navigation commands or map inputs.

Polymorph selection in metal-organic frameworks happens surprisingly early, starting at the pre-nucleation cluster stage.

Forget grid layouts: Map2World lets you generate consistent 3D worlds from arbitrary segment maps, offering unprecedented control and scalability.

Simply detecting distribution shifts in visual MBRL is easy; the real challenge is applying the right action-level corrections, which this paper tackles with a novel local expert growth strategy.

Even the most advanced language models still lose money and demonstrate unsophisticated strategies when tasked with maximizing long-term bankroll growth in a realistic sports betting simulation, highlighting a significant gap in their sequential decision-making capabilities.

Semantic rollouts and town-adversarial regularization can significantly boost zero-shot driving performance in unseen CARLA towns, even without explicit navigation commands or map inputs.

Polymorph selection in metal-organic frameworks happens surprisingly early, starting at the pre-nucleation cluster stage.

Forget per-scene optimization: GenWildSplat achieves state-of-the-art 3D reconstruction from sparse, unposed images in real-time using a purely feed-forward approach.

Achieve state-of-the-art open-vocabulary occupancy prediction without any training data, outperforming supervised and self-supervised methods by a large margin.

Control over physical properties like friction and restitution in generated videos is now possible, paving the way for more realistic and controllable video synthesis.

Today's visual generation models are often evaluated on the wrong things, leading to inflated performance claims that mask critical failures in spatial reasoning, temporal consistency, and causal understanding.

Reconstructing real-world scenes in Minecraft unlocks a customizable embodied AI playground, but only if we can solve the occupancy prediction bottleneck – and this new dataset shows we're not there yet.

Forget painstakingly programming robot interactions – ExoActor uses video generation to hallucinate plausible behaviors, then translates them into robot actions.

Forget toy tasks: scaling synthetic computer environments unlocks surprisingly effective training data for agents tackling month-long, real-world productivity workflows.

HERMES++ achieves state-of-the-art performance in both future point cloud prediction and 3D scene understanding by unifying these tasks within a single driving world model.

UAV swarms can now adapt to changing conditions and replan trajectories in real-time by inferring expert-like behaviors from a learned probabilistic world model, avoiding computationally expensive re-optimization.

Real-time robot control just got a 50x speed boost thanks to MotuBrain's efficient world action model.

Marrying short-horizon physics-based control with learned long-horizon intent yields safer and more reliable robot navigation in dense, dynamic environments.

Hyperspherical latent spaces unlock better 3D scene understanding from vision transformers, especially when bandwidth is constrained.

Finally, a reinforcement learning algorithm, PGP, can provably find near-optimal policies that respect safety and resource constraints, even when the policy space is non-convex.

Tabular foundation models can dramatically accelerate robot policy learning by enabling efficient global exploration within dynamically constructed policy subspaces.

By pretraining a VLA model with goal-conditioned RL, PRTS learns to reason about goal reachability, leading to substantial gains in long-horizon robotic tasks and zero-shot generalization.

Tackling mean-field control with common noise requires a novel integrated q-function (Iq-function) approach to identify optimal policies as fixed points.

Graph-structured world models aren't just another architecture; they're a fundamentally different paradigm for injecting relational inductive biases that could unlock more robust and interpretable AI.

A generative model of human physiology not only beats existing clinical risk scores at predicting disease, but also accurately simulates the effects of clinical interventions, paving the way for personalized medicine.

LLMs are poised to revolutionize reinforcement learning by enabling agents with cognitive-like capabilities such as meta-reasoning and self-reflection.

Understanding how charging strategies and charger types reshape both service-level outcomes and grid-facing behavior is crucial for optimizing EV charging infrastructure.

Agentic AI and digital twins can slash traffic light waiting times, outperforming traditional RL methods.

Forget tedious calibration – DOT-Sim lets you train tactile perception policies in simulation and deploy them directly to real robots with impressive accuracy, thanks to its physically accurate and rapidly calibrated model.

Forget sparse, catastrophic rewards – GSDrive uses differentiable 3D Gaussian Splatting to provide dense, physics-based feedback, dramatically improving end-to-end driving policy learning.

Orchestrating autonomous vehicles with dynamic priority scoring in marshaling yards can significantly boost throughput and prevent gridlock compared to static, isolated autonomy.

By representing deformable linear objects as a chain of relative rotations, RopeDreamer achieves state-of-the-art prediction accuracy and topological consistency in long-horizon manipulation tasks.

Kernel Sum-of-Squares optimization can guide sampling-based trajectory optimization out of local minima in high-dimensional contact-rich manipulation tasks.

Finally, a computational model quantifies driving safety by determining if a driver has a collision-free escape route, bridging a 90-year gap between theory and practice.

Autonomous vehicles can now make more judicious lane changes, improving traffic flow and safety, thanks to a federated reinforcement learning system that prioritizes urgency.

Achieve state-of-the-art 3D scene reconstruction from sparse views with 80% less training data by learning to generate, not just match, 3D structures.

Guaranteeing robot safety in cluttered environments becomes tractable by combining neural radiance fields with reachable set representations for constrained optimal control.

Quadruped robots can now nimbly navigate complex 3D terrain using only onboard depth images, thanks to a hierarchical policy that learns strategic navigation and posture adaptation.

Achieve real-time robotic action with 79-91% success while generating high-fidelity 4D reconstructions, all within a single unified world model.

VLN agents can navigate more accurately in zero-shot settings by "looking forward, now, and backward," mimicking human navigational strategies.

Rope-assisted climbing robots can now nimbly navigate complex vertical terrains thanks to a new bi-level optimization strategy that coordinates foothold selection and dynamic motion.

Rule-based high-level coaching can drastically improve the safety and sample efficiency of goal-conditioned RL agents in UAV missions, even without pretraining.

Starshaped set filtering slashes computation time and boosts robustness for robot planning in noisy environments, outperforming traditional convex optimization methods.

Robots get a spatial-temporal reasoning boost with STARRY, a world model that aligns future predictions with action generation, leading to a significant jump in manipulation success.

Forget photorealism: the next frontier for 3D generation is creating physically plausible, interactive environments that can train robots.

Achieve robust robot control through intersections by learning phase-conditioned potential functions, sidestepping the instability issues of velocity-dependent methods.

RL agents can learn to stay safe in complex environments without excessive conservatism by explicitly modeling and avoiding regions of high uncertainty.

Latent dynamics models like Dreamer can lure you into a false sense of security: their epistemic uncertainty estimates are unreliable because they're biased towards high-reward attractors in the latent space, even when the real world is different.

Achieve a \$1.22M profit in city-scale EV ride-hailing by combining semi-Markov RL with a feasibility-guaranteed MILP projection, outperforming strong baselines and eliminating feeder-limit violations.

Achieve superhuman performance in imperfect-information games by learning to exploit opponent weaknesses with a transformer-based meta-agent, without sacrificing equilibrium safety.

Neuro-symbolic guidance can dramatically accelerate reinforcement learning in sparse-reward environments, even when the symbolic knowledge is imperfect.

Randomly sampling tasks in offline RL hurts zero-shot generalization, but extracting task vectors directly from the dataset boosts performance by 20%.

Existing robotic methods falter in tackling fundamental physical reasoning challenges, as evidenced by KinDER's rigorous benchmark evaluation.

Future information in world action models isn't just a target to predict; it's a compressible correction that can be distilled for improved performance.

A novel GRU-based dynamics model for tendon-driven robots eliminates self-excited oscillations, achieving superior robustness and accuracy in control.

Prioritizing resource-aware grasps, where finger usage is explicitly modeled, dramatically improves the success rate of sequential dexterous manipulation tasks.

Robots often fail at simple household tasks not because they're confused, but because their plans become detached from the messy, ever-changing physical world.

Key contribution not extracted.

Key contribution not extracted.

Improved constraint margins under parametric uncertainty lead to tracking performance that rivals traditional NMPC methods, even in challenging aerial maneuvers.

Variational neural belief parameterizations can drastically enhance grasping success rates and reduce planning time under multimodal uncertainties, outperforming traditional methods by an order of magnitude.

The bidirectional strategy for UGV-UAV cooperation not only optimizes pathfinding in uncertain environments but also demonstrates significant time savings when leveraging multiple UAVs.

AI is poised to revolutionize protein dynamics research, but key challenges remain in ensuring scalability, thermodynamic consistency, and kinetic fidelity.

Key contribution not extracted.

LLMs are revolutionizing conversational AI research, and this survey offers a structured guide to navigating the rapidly evolving landscape of LLM-powered user simulation.

Text-to-video models can now learn geometrically consistent world dynamics via reinforcement learning, without expensive architectural changes.

AI can now pilot jet trainers to avoid ground collisions, even with limited visibility.

Discrete diffusion policies, typically used for image generation, turn out to be surprisingly effective and efficient asynchronous executors for robots acting in dynamic environments, outperforming traditional continuous control methods.

Zero-shot Sim2Real transfer for a humanoid ballbot is now possible thanks to a friction-aware RL framework and high-fidelity simulation that models omni-wheel mechanics.

Successfully backing up a trailer without jackknifing or hitting anything just got easier thanks to a new path-planning algorithm that respects the physics of articulated vehicles.

Separating geometry from logic with fuzzy path constraints yields motion planning specifications that are both more intuitive for humans and more amenable to learning from demonstrations.

Autonomous vehicles can drive more efficiently by using a new metric that links real-time acceleration decisions to overall travel time.

Multi-robot motion planning can be accelerated by over 850X, enabling solutions in milliseconds, by exploiting SIMD parallelism with vector-accelerated primitives.

Ditch expensive robot trials: a novel "betting" framework lets you accurately predict real-world robot performance using only cheap simulations.

AtomWorld achieves the previously impossible: simulating the degradation of reactor pressure vessel steel at the atomistic level across year-and-meter scales.

Achieve real-time learning-based control of complex robotic systems by exploiting differential flatness for dramatic speedups in MPC computation.

Forget slow, expensive real-world trials: dWorldEval's discrete diffusion world model lets you evaluate robot policies across thousands of environments and tasks with unprecedented speed and accuracy.

The fragmented field of world modeling can now be unified under a "levels x laws" taxonomy, revealing critical gaps in autonomous model revision and decision-centric evaluation.