Search papers, labs, and topics across Lattice.
Tsinghua AIAffiliation:Guangdong Laboratory of ArtificialHITHuaweiUSTCZJUAug 13, 2026arXiv:2608.12939This paper introduces Action-Conditioned Predictive Consistency (ACPC) as a diagnostic tool for evaluating the robustness of joint-embedding predictive architectures (JEPAs) against visual perturbations. By measuring the divergence between clean and visually perturbed histories during action-conditioned rollouts, the authors establish a framework that bounds the impact of perturbations on multi-step prediction errors and planner costs. Experiments across four visual control tasks demonstrate that ACPC effectively predicts how perturbations influence model performance, with complementary measures like Invariance Radius and Separation Rate providing additional insights into model robustness.
Visual perturbations can significantly alter predictions in world models, but ACPC offers a quantifiable way to diagnose and mitigate these effects.
Joint-embedding predictive architectures (JEPAs) learn world models that predict in a compact latent space rather than in pixels, reducing the pressure to model nuisance appearance. Yet this provides no guarantee against visual perturbations: they can still alter the encoded representation and affect subsequent action-conditioned predictions. Bisimulation captures this requirement precisely: two observations should be treated as the same state only when their action-conditioned consequences agree. Guided by this criterion, we introduce Action-Conditioned Predictive Consistency (ACPC), a diagnostic that measures how far a clean history and a visually perturbed view of it diverge after being rolled forward under the same action sequence. We prove that this divergence bounds the perturbation-induced change in multi-step prediction error and planner cost. Building on pairwise ACPC, we define two complementary measures: the Invariance Radius (IR) summarizes clean-perturbed rollout spread, while the Separation Rate (SR) checks whether different states remain distinguishable after rollout. Experiments on four visual control tasks show that pairwise ACPC predicts perturbation-induced prediction and cost changes. On LeWM, the IR-SR screen transfers across tasks, and the joint diagnostic remains informative under blur and resize. PLDM exhibits similar diagnostic trends under a different architecture.
