Search papers, labs, and topics across Lattice.
This paper introduces Anchor-Align, a novel finetuning approach for vision-language-action (VLA) models that addresses the issue of representation drift during behavior cloning (BC) by incorporating two key objectives: Vision-Language Anchoring and Language-Action Alignment. By distilling representations from a frozen pretrained model and aligning language and action predictions on the same observations, Anchor-Align significantly enhances the performance of VLA policies on real robotic tasks. Experimental results show a marked improvement in success rates on the xArm7 robot and robust performance across various out-of-distribution scenarios, demonstrating that effective representation preservation and action learning can coexist.
Anchor-Align boosts real-robot success rates from 28% to 54% by preventing representation drift during finetuning, revealing a critical synergy between preserving pretrained knowledge and effective action learning.
Finetuning a pretrained vision-language model (VLM) on robot demonstrations via behavior cloning (BC) has become the standard recipe for vision-language-action (VLA) policies. However, BC finetuning progressively overwrites the pretrained representations that support visual and semantic generalization. Co-training on web image-text data, a common remedy, does not prevent this; it applies language and action losses to separate observations, leaving VLAs with language-action misalignment that standard manipulation benchmarks do not expose. We propose Anchor-Align, which augments BC with two objectives: Vision-Language Anchoring distills layer-wise representations from a frozen VLM copy to prevent this drift, while Language-Action Alignment converts each action target into a discrete motion-direction label and jointly trains language and action prediction on the same robot observation. On a physical xArm7 robot, across two widely used VLA architectures, Anchor-Align improves real-robot success on both (28% to 54% and 37% to 60%). At scale in simulation, we demonstrate consistent improvements on OOD perturbations, perceptual robustness, and long-horizon control across LIBERO-PRO, LIBERO-Plus, and CALVIN, respectively, suggesting that preserving pretrained representations and effective action learning are not fundamentally at odds. Project page: anchoralignvla.github.io
