Search papers, labs, and topics across Lattice.
This paper introduces LeFlow, a novel approach that transforms action planning in latent world models by learning a reusable latent trajectory prior, which allows for amortized planning rather than relying on costly iterative optimization for each state-goal pair. By employing a rectified-flow model for conditional latent trajectory generation and an inverse dynamics decoder for action chunking, LeFlow significantly reduces planning time while improving success rates across various goal-conditioned pixel-control benchmarks. The findings suggest that latent world models can effectively support both prediction and reusable planning, enhancing efficiency in action planning tasks.
Amortizing planning in latent world models leads to an order-of-magnitude reduction in planning time while boosting success rates across multiple benchmarks.
Latent world models are inherently strong encoders that transform image pixel to latent embedding, yet existing world models still rely on online trajectory optimization for action planning: for every state-goal pair, an iterative optimizer is run from scratch to search for optimal action sequences, treating the world model as a black-box simulator. This approach pays the full iterative optimization cost anew at every replanning step and reuses no planning experience across queries. In this work, we ask whether planning itself can be amortized once a latent world model has been learned. We present LeFlow, which learns a reusable latent trajectory prior operating directly in the latent dynamics space from the world model. LeFlow recasts planning as conditional latent trajectory generation: a rectified-flow model imagines a future latent path between the current and goal embeddings, an inverse dynamics decoder turns latent transitions into action chunks, and the frozen world model verifies each candidate by autoregressive rollout. Across four major goal-conditioned pixel-control benchmarks, LeFlow replaces iterative action-space optimization with amortized latent planning and fixed-budget rollout selection, achieving consistent success-rate gains with an order-of-magnitude reduction in planning time. Our results argue that latent world models should support not only prediction but reusable planning priors. Our code is available at https://github.com/hsiangwei0903/LeFlow.
UW