Search papers, labs, and topics across Lattice.
This paper investigates the internal representations of vision-language-action models (VLAs) by probing the residual stream of the pretrained PaliGemma backbone to extract a linearly readable signal of task progress. The authors demonstrate that this signal, which indicates the normalized time remaining in a trajectory, is present even before training on specific robotic tasks and can generalize to unseen tasks. Additionally, they show that a simple linear probe can effectively serve as an out-of-distribution detector for stalled task progress, outperforming state-of-the-art methods in this context.
Task progress in vision-language-action models can be read directly from their internal representations, even before task-specific training, revealing a surprising depth of interpretability.
Vision-language-action models (VLAs) are moving rapidly towards deployment as general-purpose manipulation policies, but we currently lack basic tools for understanding what these models represent internally or for monitoring them at runtime. Leveraging ideas from mechanistic interpretability, we probe the residual stream of $\pi_{0.5}$ and find that task progress, the normalized time remaining in a trajectory, is linearly readable from the activations. We find that this signal is present in the pretrained PaliGemma backbone prior to training on any robot-specific data. A single linear probe generalizes to unseen tasks and varies under language counterfactuals when trained on multi-prompt data, but does not enable meaningful steering of the policy. These properties make the signal directly useful for instrumenting deployed VLAs. We use the probe as a simple label-free OOD detector, which detects stalled task progress, and find it competitive with state-of-the-art methods. Our results suggest that VLAs have rich, linearly readable internal representations of semantic quantities like task progress, and that learning to read these signals offers a lightweight, interpretable path toward monitoring deployed visuomotor policies.
