Search papers, labs, and topics across Lattice.
This paper investigates the integration of an action-conditioned Joint-Embedding Predictive Architecture (JEPA) with a model-based safety shield to enhance execution-time guarantees for heterogeneous robots. By predicting task progress and physical risk for action candidates, the proposed receding-horizon decision pipeline effectively filters inadmissible actions while maintaining high success rates in simulation. The results demonstrate that this framework not only improves task success over traditional safety shields but also reduces collision false negatives, indicating a significant advancement in robotic safety and reliability.
Coupling predictive safety with action-conditioned modeling allows heterogeneous robots to navigate complex environments with unprecedented reliability and reduced risk.
Vision-language-action policies generalize broadly but provide no execution-time guarantees; classical model-based planners respect kinematic and geometric constraints but generalize poorly. We study whether an action-conditioned Joint-Embedding Predictive Architecture (JEPA) world model can predict, before execution, both task progress and physical risk for candidate action chunks, and whether coupling these predictions to an embodiment-specific model-based safety shield yields a deployable pipeline for heterogeneous robots. We propose a receding-horizon decision pipeline: (1) a proposer produces K candidate action chunks; (2) an action-conditioned JEPA rolls each candidate forward in a frozen-encoder latent space conditioned on an embodiment embedding; (3) calibrated risk and progress heads score each rollout and report uncertainty; (4) a deterministic per-embodiment safety shield filters inadmissible candidates; (5) a fallback ladder handles empty-admissible-set cases. The learned ranking only reorders admissible candidates; enforcement guarantees come from the deterministic shield and fallback ladder. We evaluate with a pre-registered protocol in simulation (LIBERO-Long). In 600-episode configurations the full framework improved success over a shield-only baseline and reduced collision false negatives at matched recall. Deployment-efficiency measurements on target on-robot and edge accelerators are included. Real-robot experiments and an offline reranking significance test remain future work; see the paper for disclosures.
