Search papers, labs, and topics across Lattice.
This paper introduces SLS^2, a novel framework that integrates robust model predictive control (MPC) with learned latent world models to ensure safe motion planning from pixel inputs. By employing a joint-embedding world model with compact Markovian latent states, the authors achieve efficient trajectory optimization while incorporating conformal prediction to establish calibrated error bounds for safety. The evaluation demonstrates that SLS^2 significantly enhances both goal-reaching performance and safety compared to existing methods in vision-based control tasks.
Probabilistic safety constraints in motion planning can be achieved through a novel integration of learned latent models and robust MPC, outperforming traditional methods.
We present SLS^2, a framework for safe feedback motion planning from pixels using robust model predictive control (MPC) in learned latent world models. Our approach trains an action-conditioned joint-embedding world model with compact Markovian latent states, enabling efficient gradient-based trajectory optimization through learned latent dynamics. To enforce safety for the true system despite imperfect latent predictions, we inform a GPU-accelerated system level synthesis (SLS) robust MPC scheme with conformal prediction to obtain calibrated latent error bounds and robust latent-space constraint sets. We further learn and conformalize a latent constraint checker, allowing the SLS planner to impose probabilistic safety constraints during closed-loop execution. We evaluate our method on vision-based control tasks, where it improves both goal-reaching performance and safety over latent world-model and safe-planning baselines.
