Search papers, labs, and topics across Lattice.
This paper introduces ROAD-VLA, an advantage-guided self-distillation framework designed to enhance online adaptation in vision-language-action (VLA) models by addressing the challenges posed by sparse rewards. By constructing a proximal teacher in action space through perturbation of action-token logits using calibrated advantage estimates, the method provides denser supervision while aligning closely with the current policy. The results show that ROAD-VLA significantly outperforms traditional PPO methods across various robotic manipulation tasks, highlighting its effectiveness in both in-distribution and out-of-distribution scenarios.
Sparse rewards are transformed into dense supervision, enabling VLA models to adapt robustly in real-time across diverse robotic tasks.
Effective online adaptation of vision-language-action (VLA) models remains challenging, as sparse rewards provide weak supervision for high-dimensional autoregressive action policies. Although self-distillation can in principle provide denser training signals, we find that text-based privileged teachers conditioned on demonstrations, retrieved experiences, or high-level plans are ineffective for VLA adaptation, exposing a modality gap between symbolic guidance and low-level robot actions. We propose ROAD-VLA, an advantage-guided self-distillation framework that constructs a proximal teacher directly in action space by perturbing action-token logits with calibrated advantage estimates. This converts sparse rewards into dense token-level supervision while keeping the teacher close to the current policy. We further derive a policy-improvement lower bound under calibrated advantages and accurate teacher matching. Across seven robotic manipulation environments with in-distribution and out-of-distribution shifts, ROADVLA outperforms PPO in nearly all settings, demonstrating robust online VLA adaptation.
