Search papers, labs, and topics across Lattice.
This paper introduces Task-Agnostic Pretraining (TAP), a two-stage framework for Vision-Language-Action (VLA) models that separates the learning of physical competence from semantic alignment. By leveraging self-supervised learning on unlabeled interaction data, TAP achieves performance comparable to models trained on over 1 million expert trajectories while using significantly less labeled data. The results show a 10% absolute improvement over standard behavior cloning on the SIMPLER benchmark and maintain a 25% success rate in real-world scenarios under challenging conditions, highlighting the robustness of the learned representations.
Task-Agnostic Pretraining enables VLA models to achieve expert-level performance with orders of magnitude less labeled data, revolutionizing the scalability of embodied AI.
Vision-Language-Action (VLA) models are fundamentally bottlenecked by the scarcity of expert demonstrations -- triplets of observations, instructions, and actions that are costly to collect at scale. We argue that this bottleneck stems from conflating two distinct learning objectives: acquiring physical competence (how to move) and acquiring semantic alignment (what to do). Crucially, only the latter requires language supervision. Building on this Decomposition Hypothesis, we propose Task-Agnostic Pretraining (TAP), a two-stage framework that first learns transferable motor priors from cheap, unlabeled interaction data -- including discarded off-task trajectories and autonomous robot play -- via a self-supervised Inverse Dynamics objective. A lightweight second stage then grounds these priors in language using minimal expert data. On the SIMPLER benchmark, TAP matches models trained on over 1M expert trajectories while using orders of magnitude less labeled data, yielding a 10% absolute gain over standard behavior cloning. On a real-world WidowX platform, TAP retains 25% success under camera perturbations where internet-scale baselines collapse to 0%, demonstrating that task-agnostic pretraining produces robust, transferable physical representations and offers a scalable path forward for Embodied AI.
