Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper introduces a human-in-the-loop online training framework for robot flexibility in variable environments, integrating a real-time synchronized digital twin with reinforcement learning and human demonstrations. The core innovation lies in a dual-actor architecture that leverages demonstrations to guide adaptation without explicit imitation loss, enabling the robot to resume training and improve performance after workspace changes. Experimental results on a collaborative robot demonstrate significantly higher success rates compared to methods with direct imitation loss, even with suboptimal human demonstrations.
Human demonstrations, even suboptimal ones, dramatically boost robot adaptation in dynamic environments when integrated via a real-time digital twin and a dual-actor RL framework, outperforming direct imitation loss methods.
Growing automation makes collaborative robots work in more variable environments, increasing the need for adaptation. We propose a human-in-the-loop online training framework combining a digital twin (DT), reinforcement learning (RL), and human demonstrations. Unlike DTs used mainly to generate synthetic data before task execution, our DT is synchronized with the physical system in real time through camera feeds, allowing the virtual robot to update its observations and policy from real-world feedback. A dual actor framework integrates imitation learning (IL) without adding a direct imitation loss to the RL actor, so demonstrations can guide adaptation instead of manual reprogramming. The proposed framework is demonstrated on the Ufactory Xarm5 collaborative robot, where the robot's end-effector aims to reach the target position while avoiding obstacles. The experiments show that the framework can resume training after a change in the physical workspace and that, with a fixed set of non-optimal demonstrations, the dual actor framework achieves a much higher final success rate than two methods that add an imitation loss to the actor. The same pattern holds with real human demonstrations collected in virtual reality (VR): with demonstrations that never reach the goal, the dual actor framework reached 83-100% mean deterministic evaluation success, against 0-17% for the two imitation-loss methods.