Search papers, labs, and topics across Lattice.
This paper introduces HiL-ResRL, a model-agnostic fine-tuning adapter that enhances the deployment of Vision-Language-Action (VLA) models in robotic manipulation by addressing the limitations of Behavior Cloning. By training a residual policy that corrects suboptimal actions and incorporates human-in-the-loop guidance, the framework effectively mitigates distributional shifts and improves training efficiency. Experimental results show that this approach achieves over 95% success in real-world robotic tasks after just 1.5 hours of online reinforcement learning training, demonstrating its practical applicability in industrial settings.
Achieving over 95% success in real-world robotic tasks after just 1.5 hours of training, this model-agnostic framework could redefine the deployment of VLA models in industry.
Recent advancements in generative imitation learning have significantly propelled the field of robotic manipulation. However, the majority of existing models rely heavily on Behavior Cloning (BC), a paradigm that suffers from compounding errors and distributional shift. Consequently, the efficacy of these models in practical industrial deployments remains limited. To address these challenges, we introduce a novel, plug-and-play fine-tuning pipeline designed to facilitate the robust deployment of Vision-Language-Action (VLA) models in real-world environments. In contrast to contemporary reinforcement learning (RL) fine-tuning strategies, which are often constrained by specific model architectures, our proposed framework is model-agnostic and adaptable to a diverse range of VLA models. We conceptualize VLA-generated actions as a unified interface, upon which we train a residual policy. This policy is designed to rectify suboptimal actions and address the distributional shift inherent in imitation learning. Additionally, we incorporate human-in-the-loop guidance to ensure safe exploration and maximize training efficiency. We conduct experiments directly in real-world robotic settings. The results demonstrate that within only 1.5 hour of real-world online RL training, the average success rate exceeds 95% on real robots. Our work presents a practical solution for deploying behavior cloning models in industrial scenarios.
