Track genomics, neuroscience and molecular biology.
Search papers, labs, and topics across Lattice.
This paper addresses the challenge of transferring knowledge from abundant human manipulation videos to costly robot learning by analyzing causal relationships within subtasks. The proposed Causal-Beam-Guided Subtask Transfer (CBST) method identifies and retains critical contextual conditions and interaction dependencies from human demonstrations to guide robot action generation. Experiments demonstrate that CBST improves robot manipulation task success by conditioning a world-action model on causally relevant semantic information derived from human videos, without requiring direct motion correspondence.
Robots can now learn complex manipulation skills from human videos by focusing on transferable causal dependencies, bypassing the need for direct motion imitation.
Scaling robot manipulation models requires large amounts of interaction data, yet collecting diverse real-robot experience remains costly. Human manipulation videos, in contrast, provide abundant and diverse task experience that could potentially benefit robot learning. However, differences in morphology, action spaces, and control mechanisms make human motions unsuitable as direct supervision for robot actions, raising a scientific question: what knowledge in human videos should be transferred across embodiments to robot manipulation? To address this question, we analyze subtask transfer through the lens of causal beams, focusing on which contextual conditions and interaction dependencies should be retained to support local task effects. Based on this perspective, we propose Causal-Beam-Guided Subtask Transfer (CBST). CBST learns manipulation-related representations from human videos and grounds them in task-level semantic variables using robot demonstrations. Within a predefined manipulation model, causal-beam constraints guide contextual information selection by requiring retained assignments to sustain local mechanism outputs while preserving nontrivial responsiveness to retained inputs. The selected semantic information conditions a world-action model, allowing the robot to generate actions from its own observations without requiring human-robot motion correspondence. Experiments on robot manipulation benchmarks show improvements in task success over the corresponding world-action model baselines.