Search papers, labs, and topics across Lattice.
This paper investigates continual reinforcement learning (CRL) for large pre-trained Vision-Language-Action (VLA) models, challenging the assumption that complex CRL strategies are necessary to avoid catastrophic forgetting. They demonstrate that simple Sequential Fine-Tuning (Seq. FT) with LoRA surprisingly outperforms more sophisticated CRL methods across three models and five lifelong RL benchmarks. Analysis reveals that the robustness of Seq. FT stems from the synergy between large pre-trained models, parameter-efficient adaptation, and on-policy RL, effectively reshaping the stability-plasticity trade-off.
Forget complex continual learning algorithms: simply fine-tuning large vision-language-action models with LoRA achieves surprisingly strong performance in lifelong reinforcement learning.
Continual Reinforcement Learning (CRL) for Vision-Language-Action (VLA) models is a promising direction toward self-improving embodied agents that can adapt in openended, evolving environments. However, conventional wisdom from continual learning suggests that naive Sequential Fine-Tuning (Seq. FT) leads to catastrophic forgetting, necessitating complex CRL strategies. In this work, we take a step back and conduct a systematic study of CRL for large pretrained VLAs across three models and five challenging lifelong RL benchmarks. We find that, contrary to established belief, simple Seq. FT with low-rank adaptation (LoRA) is remarkably strong: it achieves high plasticity, exhibits little to no forgetting, and retains strong zero-shot generalization, frequently outperforming more sophisticated CRL methods. Through detailed analysis, we show that this robustness arises from a synergy between the large pretrained model, parameter-efficient adaptation, and on-policy RL. Together, these components reshape the stability-plasticity trade-off, making continual adaptation both stable and scalable. Our results position Sequential Fine-Tuning as a powerful method for continual RL with VLAs and provide new insights into lifelong learning in the large model era. Code is available at github.com/UT-Austin-RobIn/continual-vla-rl.
