Search papers, labs, and topics across Lattice.
This paper introduces EvoHIL, a novel framework for human-in-the-loop reinforcement learning that addresses three critical limitations: static visual reward models, temporally inconsistent actions, and sensitivity to appearance changes. By employing a self-evolving reward model, Action Flow Stabilization for coherent action generation, and retention-aware offline fine-tuning, EvoHIL significantly enhances the performance of robots in contact-rich manipulation tasks. Experimental results demonstrate that EvoHIL outperforms traditional human-in-the-loop and imitation learning baselines in terms of task success, motion smoothness, and alignment with human feedback across various manipulation scenarios.
EvoHIL achieves unprecedented improvements in robotic manipulation by dynamically adapting reward models and ensuring action coherence, setting a new standard for human-in-the-loop learning.
Human-in-the-loop reinforcement learning (HIL-RL) enables robots to learn contact-rich manipulation from limited real-world interaction, but deployment exposes three coupled limitations: static visual reward models fail under scene changes; independently sampled actions cause temporally inconsistent motion; and vision-based policies remain sensitive to appearance shifts. We present EvoHIL, a unified framework that adapts the reward model, action generator, and visual do main within a staged human-in-the-loop learning process. First, self-evolving reward (SER) adapts the success classifier from human-confirmed positives and provisional weak negatives. Second, Action Flow Stabilization (AFS) generates temporally coherent action chunks through flow matching, grounding policy updates in executed action prefixes and demonstrated behavior. Third, retention-aware offline fine-tuning replays relit interaction data while anchoring the AFS actor-critic to prior behavior, adapting the visual domain without additional robot interaction. Across six manipulation tasks on Franka FR3 and SO-101 arms under a controlled lighting shift, EvoHIL improves task success, agreement with human-confirmation labels, motion smoothness, and completion time relative to human-in-the-loop and imitation baselines.Project page: https://anonymous4366.github.io/EvoHIL/
