Search papers, labs, and topics across Lattice.
This paper introduces UniIntervene, an agentic intervention model designed to enhance human-in-the-loop reinforcement learning (HiL-RL) by autonomously managing unproductive exploration in robotic manipulation tasks. By employing future-conditioned action-value estimation and a temporal value-risk critic, UniIntervene significantly reduces the need for human corrections, allowing for a more scalable and efficient learning process. Experimental results show that UniIntervene increases the average success rate by 8.6% while decreasing human interventions by 57% compared to existing HiL-RL methods.
Autonomous intervention can cut human oversight in reinforcement learning by over half while boosting success rates in real-world tasks.
Human-in-the-loop reinforcement learning (HiL-RL) has emerged as an effective paradigm for real-world robotic manipulation, enabling online policy improvement with human guidance. However, current HiL-RL frameworks remain intervention-intensive, relying on frequent human corrections to redirect the policy out of unproductive exploration, which incurs high labor cost and limits real-world scalability. To address this, we propose UniIntervene, an agentic intervention model that detects unproductive exploration and autonomously recovers the policy toward high-value states, taking over the bulk of interventions from human operators. Specifically, UniIntervene first performs future-conditioned action-value estimation, predicting the latent consequence of the current action and evaluating its induced value, which provides a more stable progress signal. Building on this, a temporal value-risk critic aggregates recent value dynamics and triggers intervention when the estimated value exhibits sustained stagnation or degradation. When intervention is required, UniIntervene retrieves a high-value recovery target from a memory of past intervention episodes and produces executable corrective actions through a goal-conditioned recovery policy. In this way, UniIntervene turns intervention from passive human correction into a value-aware recovery process for efficient real-world RL. Extensive experiments on diverse real-world manipulation tasks demonstrate that UniIntervene improves the average success rate by 8.6% while reducing human interventions by 57% relative to state-of-the-art HiL-RL baselines.
