Search papers, labs, and topics across Lattice.
This paper introduces FF-JEPA, a hierarchical planning approach that enhances long-horizon planning in world models by utilizing two forward dynamics models: a standard action-conditioned model and an action-free latent planner. By eliminating the need for explicit goal images and decomposing complex trajectories into manageable subgoals, FF-JEPA effectively addresses the computational inefficiencies of traditional methods like the Cross-Entropy Method (CEM). Preliminary results on the PushT environment indicate that FF-JEPA significantly mitigates long-horizon collapse, showcasing its potential for goal-free planning in real-world scenarios.
FF-JEPA transforms long-horizon planning by enabling goal-free trajectory optimization without the need for explicit goal images.
Joint Embedding Predictive Architectures (JEPAs) have shown promising world modeling capabilities, enabling planning in latent space by optimizing action trajectories using methods like the Cross-Entropy Method (CEM). These methods are, however, too computationally expensive and ineffective for long-horizon planning. Furthermore, these methods typically require an explicit image of the goal state, which is not always possible in real-world tasks. In this work, we tackle these limitations by proposing Forward-Forward-JEPA (FF-JEPA), a hierarchical approach leveraging two forward dynamics models. Alongside a standard action-conditioned forward model, we introduce an action-free latent planner that predicts the next subgoal given the current state. This approach removes the need for goal images and enables long-horizon planning by decomposing complex trajectories into a sequence of tractable, short-term optimization problems. Preliminary results on PushT demonstrate that FF-JEPA successfully overcomes flat world models' long-horizon collapse, highlighting this approach as a promising direction for goal-free planning.
