Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
World-model planners trained with a specific plan-to-motion mapping struggle when a different controller executes plans, creating a planning-execution gap. Directly substituting predicted executions yields minimal gains due to a semantic mismatch between reference plans and post-controller motion. ExecutionBridge adapts the planner's latent rollouts using inferred controller-specific execution characteristics, significantly improving planning performance across unseen controllers without modifying the frozen planner or world model.
A frozen world-model planner's performance can be drastically improved by adapting its latent rollouts to controller-induced execution shifts, rather than directly substituting predicted trajectories.
World-model planners evaluate candidate trajectories by predicting their future consequences, implicitly relying on the plan-to-motion relationship learned during training. At deployment, a different downstream controller may execute the same planned trajectory differently, creating a planning-execution gap between the frozen model's imagined future and actual execution. A natural remedy is to replace the planned trajectory with its predicted or observed execution. However, we find that better execution fidelity does not necessarily yield better decisions: directly substituting predicted execution trajectories yields only a 0.048 PDMS improvement across 10 unseen controller configurations, far smaller than the gain from our proposed adaptation. This suggests a semantic mismatch between a planned trajectory, which serves as a reference for downstream control, and an executed trajectory, which already describes the resulting post-controller motion. We introduce ExecutionBridge, which infers controller-specific execution characteristics from planned-executed trajectory pairs and uses them to adapt the native ego latent rollout while keeping the original planned trajectory unchanged. The planner, world model, and reward head remain frozen. On NAVSIM, ExecutionBridge consistently improves planning across unseen controller configurations and multiple frozen world-model planners, with substantially larger gains in challenging driving scenes where execution mismatch more strongly affects candidate evaluation.