Search papers, labs, and topics across Lattice.
This paper introduces EgoRecovery, a co-training framework that leverages egocentric human demonstrations to enhance the failure recovery capabilities of embodied robots. By efficiently capturing diverse recovery behaviors through human operators, the method generates over 10 times more valid recovery data compared to traditional robot teleoperation. The results demonstrate that EgoRecovery significantly improves the success rate of robots recovering from failures, outperforming existing approaches that rely solely on robot-generated data or direct human-robot co-training.
Human demonstrations can yield over 10x the recovery data for robots, dramatically enhancing their ability to recover from failures in real-world tasks.
Robust embodied robots should be able to recover from failures and retry tasks in order to operate reliably in unstructured and noisy real-world environments. Achieving this capability requires training policies on data that captures recovery behaviors. However, collecting such data through robot teleoperation is difficult to scale, as it is time-consuming to induce diverse failure states, perform corrective actions, and reset the environment. This challenge is further exacerbated by the high diversity of failure modes, which demands substantially more recovery data than success demonstrations. In this work, we show that egocentric human data capturing failure recovery processes provides a scalable alternative. By efficiently arranging task-level failure configurations and recording short recovery segments, human operators can generate more than 10x as much valid recovery data per hour compared to robot teleoperation under our protocol. To address the embodiment gap between human and robot, we propose EgoRecovery, a co-training framework for learning recovery behavior, where human recovery demonstrations are aligned to a compact corrective-intent space shared with robot data, which captures the timing and magnitude of correction. Only a small number of robot recovery demonstrations are required to connect this intent to executable robot actions. At deployment, a learned recovery gate predicts when correction is needed from robot observations and activates the corrective intent only in recovery states. Experiments on real-world recovery tasks show that EgoRecovery improves success from failure starts over robot-only recovery, direct co-training with human recovery data, and direct intent-transfer baselines.
