Search papers, labs, and topics across Lattice.
The paper introduces WAM-TTT, a novel test-time training framework that enables robot foundation models to adapt to new task variants by leveraging raw human videos without requiring additional robot demonstrations or task-specific fine-tuning. By employing a self-supervised video prediction approach, WAM-TTT integrates human demonstrations into a lightweight adaptive memory within a frozen world action model, aligning human behaviors with robot actions through a meta-training stage. Experimental results demonstrate that WAM-TTT significantly outperforms existing in-context conditioning methods across various manipulation tasks, showcasing its efficiency and generalization capabilities.
WAM-TTT allows robot models to adapt to new tasks using only raw human videos, eliminating the need for additional demonstrations or fine-tuning.
Steering robot foundation models (RFMs) toward new task variants or user-preferred behaviors remains challenging, often requiring additional robot demonstrations, task-specific fine-tuning, or long-context conditioning. We present WAM-TTT, a test-time training framework for steering world action models from raw human videos. Rather than treating human videos as trajectories to imitate, WAM-TTT absorbs them into a lightweight adaptive memory inside a frozen WAM through self-supervised video prediction. To make this memory useful for control, we introduce a meta-training stage that aligns human demonstrations with robot behaviors using paired human-robot data and a key--value memory reconstruction objective. At test time, only unlabeled human videos are required to adapt the memory, while the pretrained WAM remains frozen. This enables efficient and reusable steering without robot actions, human-side annotations, or task-specific fine-tuning, while preserving the generalization ability of the foundation model. Extensive experiments show that WAM-TTT consistently outperforms in-context human-video conditioning baselines across diverse manipulation tasks and generalization settings.
