Search papers, labs, and topics across Lattice.
This paper introduces a framework that leverages egocentric walking videos to enhance mobile robot navigation through imitation learning. By estimating camera motion from human videos and converting it into compatible action representations, the authors enable a vision-language-action (VLA) model to jointly train on both human-derived and robot-collected datasets. The key finding reveals that this co-training approach significantly improves language understanding and action generation, outperforming models trained on individual data sources in a fruit-search navigation task.
Human egocentric videos can be transformed into a scalable training resource, boosting mobile robot navigation performance beyond traditional datasets.
Vision-language-action (VLA) models are promising for diverse robotic tasks, but their performance heavily depends on large-scale high-quality training data, whose collection on real robots is costly and time-consuming. While prior work has explored augmenting manipulation datasets with egocentric human videos, applying such approaches to mobile robot navigation remains challenging due to viewpoint changes during locomotion. In this paper, we propose a framework that converts egocentric walking videos into datasets for mobile robot imitation learning. The proposed method estimates camera motion from human videos and transforms it into action representations compatible with ground mobile robots. By jointly training a VLA model on human-derived and robot-collected datasets, the model achieves improved language understanding and more robust action generation than training with either data source alone. Experiments on a fruit-search navigation task demonstrate that human egocentric videos provide an effective and scalable data source for mobile robot learning.
