Search papers, labs, and topics across Lattice.
This paper investigates the concept of decision-metric alignment in latent world models for model-predictive control (MPC) by introducing metrics that evaluate the alignment between latent and real task progress. The authors propose the Plan-Real Spearman and CEM-stage Spearman metrics to assess how well latent distances correspond to actual task performance, revealing critical factors like encoder distortion and terminal rollout error that affect this alignment. Their enhanced model, DA-LeWM, incorporates inverse-dynamics and demonstration-conditioned objectives, leading to faster convergence and improved online success rates compared to traditional latent world models, while maintaining similar probe scores.
Action-conditioned objectives can significantly enhance the effectiveness of Euclidean-cost MPC by aligning latent representations with real task progress.
JEPA-style latent world models can use Euclidean distance to a goal latent as the cost for model-predictive control (MPC). Strong decoding of task variables, however, does not guarantee that this particular cost ranks candidate action sequences by real task progress. We call the latter property \emph{decision-metric alignment}. We introduce Plan-Real Spearman, which measures latent--real rank agreement on random plans, and CEM-stage Spearman, which measures the same agreement as cross-entropy-method (CEM) search concentrates its proposal. We analyze sufficient conditions under which latent distance preserves real-cost rankings, identifying encoder distortion, terminal rollout error, and candidate margins as the controlling quantities. Guided by the observed empirical alignment gap, DA-LeWM augments LeWM with inverse-dynamics and demonstration-conditioned goal-action heads. Across all our experiments, DA-LeWM accelerates convergence and achieves higher online success than LeWM, while probe scores remain similar. These results show that action-conditioned objectives improve the geometry used by Euclidean-cost, CEM-based latent MPC.
