Search papers, labs, and topics across Lattice.
This paper introduces EXIMO, an innovative algorithm designed to finetune vision-language-action (VLA) policies for robotic manipulation tasks efficiently. By integrating a vision language model (VLM) that decomposes complex tasks into manageable components, EXIMO significantly reduces the reliance on extensive teleoperation datasets and enhances sample efficiency through a structured three-stage process: explore, imitate, and optimize. Experimental results demonstrate that EXIMO outperforms traditional methods in both sample efficiency and overall performance, addressing a critical challenge in real-time robotic learning.
EXIMO achieves unprecedented sample efficiency in robotic policy finetuning by leveraging a vision language model to decompose complex tasks.
How to efficiently finetune robot policies to learn new tasks on the fly? State of the art robotic manipulation policies are based on behaviour cloning of large vision-language-action (VLA) models with billions of parameters on huge teleoperation datasets. While this simple approach has enabled significant advances for robotic manipulation, finetuning of VLA policies for learning new tasks still remains an open problem. In particular, collecting teleoperation datasets requires hundreds of hours of expensive human labour and the alternative, reinforcement learning (RL), can be notoriously sample-inefficient especially for long-horizon tasks. In addition, RL with VLAs imposes several challenges due to the model's size and architectural design. In this work, we propose EXIMO, an efficient algorithm for finetuning of VLA policies. EXIMO operates in three stages: explore, imitate, and optimize. During the explore phase, EXIMO equips the VLA with a vision language model (VLM) that acts as a planner. The VLM thinks and breaks down challenging long-horizon problems into shorter ones for the VLA. The VLM, together with the VLA, is used to collect an orchestrated dataset on new tasks. During the imitate phase, the VLA is finetuned with the orchestrated data. Finally, during the optimize stage, we use residual off-policy RL to further finetune the policy. In our experiments, we ablate all three stages of EXIMO and show that it outperforms existing approaches significantly in terms of sample-efficiency and final performance.
