Search papers, labs, and topics across Lattice.
This paper introduces an active continual learning framework for Vision-Language-Action (VLA) models, addressing the limitations of passive imitation learning by utilizing uncertainty-guided data collection for more efficient fine-tuning. The authors find that while this approach enhances adaptation efficiency, focusing solely on actively-collected recovery data can lead to catastrophic forgetting of previously learned behaviors. Through empirical evaluation of techniques like replay-based data mixing and elastic weight consolidation, the study highlights the trade-offs between adapting to new data and retaining existing knowledge in large robot policies.
Active data collection can significantly boost the efficiency of fine-tuning VLA models, but beware鈥攖he wrong focus can lead to catastrophic forgetting.
Vision-Language-Action (VLA) models are commonly fine-tuned through passive imitation learning, where additional demonstrations are collected for tasks where the policy performs poorly. This approach incurs several downsides: it requires the robot to fail before data collection is triggered, provides little guidance about which states require supervision, and wastes demonstrator effort on redundant parts of the task where the policy already performs well. In this paper, we propose an active, continual learning paradigm for VLAs. We demonstrate that active, uncertainty-guided data collection leads to more efficient fine-tuning than when using passively-collected demonstrations. However, we also find that fine-tuning only on actively-collected recovery data leads to catastrophic forgetting. We evaluate techniques for continual learning, including replay-based data mixing and elastic weight consolidation, and identify tradeoffs between plasticity to uncertainty-guided recovery data and retention of previously learned behaviors. Overall, our work contributes an empirical study of active continual learning for autoregressive VLAs, establishing that uncertainty-guided recovery demonstrations can improve adaptation efficiency while also revealing open challenges when targeted new data is incorporated into large robot policies.
