World Models & Planning - Weekly Roundup

Continuous, decomposed visual verification unlocks robust rule-governed generative world modeling, outperforming binary critics and enabling structured reasoning agents.

Spatial memory via frustum slab routing slashes video world model inference time by over 4x without sacrificing generation quality.

Explicitly supervising physical state changes, not just language or latent representations, unlocks more robust VLA model generalization and real-world robotic control.

LLMs can now predict future 3D scenes independent of camera viewpoint, enabling consistent scale and cross-view querying without per-frame alignment.

Robust RL policies for adaptive cancer therapy achieve superior performance and generalization by treating patients as hidden contexts, outperforming existing methods even with noisy biomarkers and shifting disease dynamics.

Continuous, decomposed visual verification unlocks robust rule-governed generative world modeling, outperforming binary critics and enabling structured reasoning agents.

Spatial memory via frustum slab routing slashes video world model inference time by over 4x without sacrificing generation quality.

Explicitly supervising physical state changes, not just language or latent representations, unlocks more robust VLA model generalization and real-world robotic control.

LLMs can now predict future 3D scenes independent of camera viewpoint, enabling consistent scale and cross-view querying without per-frame alignment.

Robust RL policies for adaptive cancer therapy achieve superior performance and generalization by treating patients as hidden contexts, outperforming existing methods even with noisy biomarkers and shifting disease dynamics.

Wireless signals, unhindered by darkness or walls, can reconstruct building layouts by leveraging a diffusion model over physics-grounded representations of channel measurements.

Offline-trained neural networks are mathematically proven to diverge in closed-loop control, but bounded online adaptation offers a practical solution.

Real-world robot demonstrations can now be collected at 66 demos/hour by optimizing TAMP solution quality for behavior cloning.

Agents can now learn reusable knowledge from experience by converting it into ontological axioms, enabling LLMs to dynamically propose and verify hypotheses for complex tasks.

Active policy selection, not just expert samples, is essential for strategically identifying rewards in multi-agent games, enabling near-optimal downstream decision-making.

Adaptive planning horizons emerge naturally from policy confidence, outperforming fixed discount factors on challenging credit assignment problems.

Rational synthesis for multi-agent systems with unknown dynamics and $\mathsf{PLTL}_f$ objectives is achieved by integrating solution concepts like Nash equilibrium with temporal logic satisfaction.

Bidirectional future graphs allow later evidence to retroactively correct earlier motion prediction segments, achieving state-of-the-art forecasting accuracy without LiDAR or ensembling.

Humanoid robots can now grasp and manipulate bulky objects using their entire bodies, achieving 90% success in complex repositioning tasks.

A VLM can learn to dynamically "imagine" missing visual evidence to solve complex spatiotemporal reasoning tasks, outperforming direct reasoning and fixed imagination pipelines.

Offline risk-sensitive RL is now provably efficient, with new algorithms and tight bounds for dynamic and static optimized certainty equivalents.

Learning to predict the visual consequences of actions significantly boosts policy performance and generalization, while simultaneously learning to generate actions sharpens world models.

Robot policies learn 4x faster by co-evolving their training environments with a critic that diagnoses and recreates failure modes.

A single, stage-specific objective per expert in a decoupled diffusion distillation framework enables state-of-the-art video generation with a 10,000x reduction in training cost.

Persistent interactive worlds that evolve with history and agent actions are now possible through a novel coupling of a computable world engine and egocentric audiovisual generation.

Most social LLMs fail to achieve verifiable success in coordination tasks, even when rated highly by conversational judges.

Language agents can achieve superior sample efficiency and performance by jointly learning to act and forecast multi-step action sequences within a single policy backbone.

Driving models can achieve near-perfect performance with just one denoising step by learning to predict visual motion directly, outperforming multi-step prediction models and achieving 3.29x inference speed.

Autonomous driving VLAs can be made conflict-aware by dreaming world models that strictly follow navigation signals, revealing unsafe instruction scenarios.

Agentic robots are no longer just predicting actions, but actively orchestrating their own learning and execution through goal-directed reasoning.

A novel two-axis framework unifies the study of multimodal agent memory, revealing distinct tiers of memory adaptation from static reuse to self-evolving policies.

Simple feature propagation and label smoothing on relational graphs match much of the performance of complex GNNs, suggesting architectural complexity might be over-emphasized.

Budgeted adaptation under opponent shift can be solved by dual-counterfactual control that explicitly balances immediate adaptation value against future adaptation loss and local strategic risk.

Learning optimal policies from a single RL trajectory can be as sample-efficient as unrestricted exploration.

Decoupled Q-learning achieves state-of-the-art performance on continuous control by adaptively growing action grids, but surprisingly, the adaptive growth itself is not the key driver of success.

Continual learning for molecular communication achieves robust performance estimation across evolving experimental complexities by intelligently synthesizing past data.

Learned representations on a hypersphere unlock closed-form planning and adaptive waypoint generation, enabling complex manipulation from scratch where baselines fail.

Average-reward option Q-learning achieves $\tilde{O}(1/\text{iterations})$ convergence, unlocking continual RL tasks previously intractable for option-based methods.

Foundation models trained on synthetic order flow perform amortized Bayesian inference, decoding latent market regimes with surprising accuracy and outperforming traditional methods for simulator calibration.

Few-shot behavior cloning for autonomous driving achieves robust traffic light compliance by adaptively weighting demonstrations based on action density.

Robot policies can now explicitly track and act upon physical time, unlocking robust performance in tasks requiring precise temporal reasoning and long-term memory.

Frontier multimodal models are beginning to predict future world states, but current benchmarks reveal vast performance disparities and highlight the critical role of underlying language model capabilities.

A single latent cache can unify diverse visual representations and spatiotemporal queries for efficient, consistent dynamic world modeling across multiple video generation tasks.

Agents can now recursively evolve their own training environments, jointly building and validating tasks, contexts, and verifiers for robust long-horizon learning.

Generative models, when framed as proposal distributions for planning, unlock efficient long-horizon reasoning in model-based RL, outperforming diffusion policies and model-free methods.

A single robot world foundation model predicts dynamics and controls diverse robot bodies with top-tier accuracy and zero-shot generalization.

Generative 3D reconstruction now scales to entire indoor scenes, producing coherent, unified meshes with state-of-the-art fidelity from unposed images.

A lightweight latent world model directly conditions VLA action generation on predicted future consequences, boosting real-world robotic success rates from 38.3% to 73.3% without pixel-level video prediction.

Fine-tune robot simulation physics with unprecedented interactive control, enabling AI agents to learn from real-world interactions and achieve accurate real-to-sim transfer.

Robots can't truly improve themselves yet; current systems lack the crucial ability to recursively refine their own learning processes based on physical experience.

Offline Q-learning can achieve $\widetilde{O}(1/ε)$ sample complexity in finite-horizon KL-regularized MDPs, even without explicit pessimism, by leveraging KL regularization.

A single unified framework achieves state-of-the-art performance in both action-conditioned world modeling and real-world robotic manipulation by dynamically routing specialized experts.

A grid-based world model can achieve 89% out-of-view object recall and outperform a Kalman tracker in view, but a node-based approach offers drastically superior memory efficiency and robustness to unmodeled velocities.

Manufacturing AI can now learn to avoid costly negative transfer by selectively leveraging prior knowledge, achieving 20% lower carbon footprint prediction error with minimal target data.

Deterministic forecasting embedded within generative models is unstable; DV-Flow decouples it as a parallel velocity, unlocking stable end-to-end training and superior probabilistic forecasts.

A theoretically grounded EI algorithm for continuous simulation optimization achieves simple-regret guarantees by learning unknown heteroscedastic noise and optimizing incumbent/recommendation strategies based on regret bounds.

A novel entropy-based framework reveals hidden nonlinear predictive couplings that traditional error-magnitude metrics miss, even in stochastic, partly non-invertible settings.

Synthesized interactive work environments enable LLMs to achieve state-of-the-art performance on complex agentic benchmarks, outperforming larger models with a 27B parameter model.

A novel "Predictive Intent" token bridges VLMs and VGMs for robot manipulation, unlocking superior out-of-distribution generalization without embodied pretraining.

Agents trained on procedurally generated, harder environments significantly outperform those trained on original benchmarks, even with identical compute and optimization.