Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ActiveMimic, a novel pretraining framework that leverages egocentric human video by modeling camera motion as a viewpoint action and jointly learning active perception and manipulation. The authors demonstrate that by incorporating active perception behavior, which is often treated as noise in standard pipelines, their approach significantly enhances performance on robotic tasks compared to traditional methods. Empirical results show that ActiveMimic not only outperforms baselines pretrained on human video but also matches the performance of state-of-the-art models trained on robot data, highlighting the importance of active perception in robot learning.
ActiveMimic reveals that leveraging active perception from egocentric videos can close the performance gap with robot-pretrained models, transforming how we approach robot learning.
Egocentric human video offers a scalable alternative to robot data for pretraining, yet models pretrained on such video consistently underperform those pretrained on robot data. We attribute this gap to a missing signal, the active perception behavior in egocentric videos, where humans continuously reposition their viewpoint during manipulation, inducing camera motion that standard pipelines treat as noise. To address this, we present ActiveMimic, a pretraining framework that recovers synchronized camera and wrist trajectories from a single body-worn RGB camera, models camera motion as a viewpoint action, and jointly learns active perception and manipulation from in-the-wild egocentric human video before adapting to a target robot. Empirically, real-world experiments across tasks with diverse active perception demands show that ActiveMimic consistently surpasses baselines pretrained on human video and matches state-of-the-art models pretrained on robot data. Further analysis provides evidence that active perception capability originates from egocentric human video pretraining rather than robot-specific fine-tuning, confirming active perception as the key to unlocking egocentric human video for robot pretraining.