Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates failure modes in Vision-Language-Action (VLA) agents, comparing discrete (VQ-BeT) and continuous (Diffusion Policy, ACT) action space architectures on robot manipulation tasks. They find that direction reversal is a universal failure predictor, while jerk is only predictive for discrete architectures, and velocity monitoring is largely ineffective, especially for continuous action spaces. The study highlights the need for architecture-specific action monitoring strategies, enabled by their SafeContract toolkit.
The most common safety mechanism in VLA deployment code, velocity checking, provides effectively zero predictive signal for continuous action space VLAs, exposing a critical mismatch between current safety practices and VLA architecture.
We discover that VLA architectures fail in fundamentally different, predictable ways at the motor-command level. Running VQ-BeT, Diffusion Policy, and ACT on identical evaluation protocols (n=450 episodes across PushT and ALOHA 14-DOF bimanual manipulation), we find: (1) direction reversal rate is a universal failure predictor across all three architectures (AUROC=0.93, 0.79, 0.91; p<0.001); (2) jerk monitoring is predictive only for discrete-token architectures, following a discrete-to-continuous gradient (0.88, 0.69, 0.41); (3) velocity violations alone are non-predictive everywhere (AUROC 0.41-0.69), yet velocity checking is the most common safety mechanism in VLA deployment code; and (4) for continuous-family VLAs, velocity monitoring provides effectively zero predictive signal (AUROC=0.52 on ACT, 0.41 on Diffusion), proving that architecture-matched monitor selection is essential. These results quantify a monitoring consequence of the well-known discrete/continuous VLA distinction: the two families produce qualitatively different failure signatures that require different monitors. No single monitor works universally; architecture-matched selection is required. This finding was enabled by SafeContract, a training-free, black-box action monitoring toolkit with conformal calibration. Code: https://github.com/krishnam94/vla-edge