Search papers, labs, and topics across Lattice.
AdaJEPA introduces an adaptive latent world model that integrates test-time adaptation into model predictive control (MPC) to enhance planning accuracy in dynamic environments. By utilizing observed next-state transitions as self-supervised signals, the model recalibrates itself continuously during execution, addressing the limitations of traditional frozen latent models. Experimental results demonstrate that AdaJEPA significantly improves planning success across various goal-reaching tasks, achieving this with minimal computational overhead through just one gradient step per replanning iteration.
Adaptive latent models can recalibrate in real-time, boosting planning success rates even in shifting environments.
Latent world models enable planning from high-dimensional observations by predicting future states in a compact latent space. However, these models are typically kept frozen at test time: when their predictions become inaccurate, planning can fail, especially under test-time distribution shift. To address this, we propose AdaJEPA, an adaptive latent world model that performs test-time adaptation within the closed loop of model predictive control (MPC). After training, AdaJEPA plans and executes the first action chunk, uses the observed next-state transition as a self-supervised adaptation signal, and replans with the updated model. This closed-loop update continuously recalibrates the world model without additional expert demonstrations. Across a range of goal-reaching tasks, AdaJEPA substantially improves planning success with as few as one gradient step per MPC replanning step.
Meta AI