Search papers, labs, and topics across Lattice.
This paper introduces GUARD, a novel method for detecting failures in diffusion-based vision-language-action (VLA) policies by assessing their grounding in visual and language evidence without altering the pretrained models. By ablating salient entries in the vision-language model's key-value cache and comparing the resulting denoising responses, GUARD generates a diagnostic stream that includes metrics like sensitivity and grounding efficiency. Evaluated across five policy-benchmark settings, GUARD outperforms existing runtime monitors, achieving a notable improvement in ROC-AUC scores on unseen tasks, thus demonstrating its effectiveness as a transferable failure detection tool.
GUARD reveals that probing action-head dependence on multimodal evidence can significantly enhance failure detection in VLA policies, outperforming existing methods in unseen task scenarios.
Diffusion-based vision-language-action (VLA) policies can generate plausible actions even when their predictions are weakly grounded in the visual and language evidence defining the task. We introduce GUARD, a test-time failure detection method that measures this grounding without modifying the pretrained policy. GUARD estimates the influence of token-indexed entries in the final vision-language model key-value (KV) cache, constructs counterfactual caches by ablating salient KV entries, and compares their denoising responses with the original conditioning. Based on the comparison, we derive GUARD diagnostic stream including sensitivity, attention entropy, modality bias, and grounding efficiency, which are calibrated online and processed by a lightweight temporal classifier. We evaluate GUARD under task-held-out splits across five policy-benchmark settings, using Pi0, SmolVLA, and Alpamayo-1.5 on LIBERO, SimplerEnv, MetaWorld, and PhysicalAI-AV. GUARD achieves the best ROC-AUC on four of five unseen-task settings and ranks second on the remaining setting, improving the average unseen-task ROC-AUC by 5.73 percentage points over the strongest competing runtime monitor while remaining within 0.19 points of the best seen-task average. These results show that directly probing action-head dependence on multimodal evidence provides a transferable failure signal across policies, tasks, embodiments, and domains.
