Search papers, labs, and topics across Lattice.
UniIntervene++ addresses the challenge of evolving assistance needs in online RL by adaptively allocating control between autonomous execution and heterogeneous assisted behaviors. It formulates the RL policy, trajectory correction, and a CodePolicy as Options within a unified semi-Markov decision process, learning their relative values online. This adaptive approach, which includes competence-adaptive intervention and coupled experience learning, significantly improves success rates and drastically reduces human intervention in real-world manipulation tasks.
Real-world robots can now learn faster and require drastically less human help by adaptively deciding *when* and *how* to be assisted, rather than relying on fixed rules.
Online reinforcement learning (RL) enables robot policies to improve through physical interaction, but the assistance they require changes as their competence evolves. Existing intervention strategies based on offline estimates or fixed decision rules can therefore become mismatched to the current policy. To address this, we propose UniIntervene++, an adaptive intervention agent that learns to allocate control between autonomous execution and heterogeneous assisted behaviors during online RL. Specifically, UniIntervene++ first formulates the evolving RL policy, trajectory correction, and a task-structured CodePolicy as Options in a unified semi-Markov decision process and learns their relative values online. Building on this, competence-adaptive intervention periodically probes the RL policy through unassisted execution, keeping control allocation responsive to its evolving capability. Finally, coupled experience learning allows assisted behaviors to improve the RL policy, whose evolving outcomes in turn reshape future intervention decisions. In this way, UniIntervene++ jointly determines when to intervene, how to intervene, and when to return control as the RL policy improves. Across five real-world manipulation tasks, UniIntervene++ achieves an average success rate of 89.67%, outperforming all baselines by at least 6 percentage points, while reducing human intervention to 0.77%, a relative reduction of at least 94.6% from the best baseline. Code is available in our \href{https://github.com/dannyyudong/An-Adaptive-Intervention-Agent-for-Efficient-Real-World-Reinforcement-Learning}{GitHub repository}.
