Search papers, labs, and topics across Lattice.
This paper investigates how to best initialize Vision-Language-Action (VLA) models using pretrained Vision-Language Models (VLMs), focusing on embodied VQA supervision, parameter update strategies (LoRA vs. full finetuning), and robot-data pretraining. The study reveals that the original pretrained VLM representation is crucial for action performance, and that LoRA-based updates are more reliable than full finetuning, suggesting that preserving the pretrained VLM representation is key. Robot-data pretraining further enhances VLA initialization, especially when combined with staged LoRA training.
Over-eagerly adapting VLMs for robot control can actually hurt performance, suggesting the original VLM representation already encodes surprisingly useful action priors.
Vision-Language-Action (VLA) models widely adopt pretrained Vision-Language Models (VLMs) as policy backbones, yet it remains unclear what kind of pretrained VLM representation is useful as a VLA initialization. In this paper, we study VLA initialization as a controlled representation-design problem along three axes: capability-level embodied VQA supervision, parameter-update strategy, and robot-data pretraining. Our experiments show that the original pretrained VLM representation is a key source of action performance. However, embodied VQA adaptation does not yield uniform gains: its benefit depends on downstream bottlenecks, and gains from different capability domains are not simply additive. For update strategy, LoRA provides a more reliable initialization than Full Finetune, indicating that overly reshaping the pretrained representation can weaken VLA initialization. Robot-data pretraining further improves VLA initialization, with the strongest variant obtained by staged LoRA-based training. Together, these findings suggest that effective VLM-to-VLA adaptation should inject action-relevant embodied and robot-trajectory signals while preserving the pretrained VLM representation that remains useful for action learning.
