Search papers, labs, and topics across Lattice.
This paper introduces CORE, a novel policy learning framework that leverages action-free visual demonstrations to enhance robot manipulation by focusing on Common Outcome Regularities. By training a terminal outcome encoder and utilizing visual goal prototypes, CORE effectively bridges the embodiment gap and provides concrete geometric constraints for task completion. The framework demonstrates significant improvements in success rates across various environments, outperforming traditional text-conditioned methods by notable margins.
Visual goal prototypes can boost robot manipulation success rates by up to 17% compared to text-based instructions, revealing the power of leveraging action-free demonstrations.
Robot imitation learning often relies on costly robot demonstrations, while abundant action-free visual demonstrations, such as human videos, are difficult to use because they lack robot-executable actions and suffer from embodiment gaps. We propose CORE, a policy learning framework that extracts Common Outcome Regularities from visual demonstrations. Rather than transferring explicit actions across embodiments, CORE exploits a key observation: although successful trajectories for the same task can be diverse, their terminal states often share stable object configurations, spatial relations, and contact constraints. CORE first trains a terminal outcome encoder with contrastive and auxiliary temporal objectives, then aggregates successful terminal embeddings into visual goal prototypes, and finally injects these prototypes as global goal conditions into robot policies. Compared with language instructions, visual goal prototypes provide more concrete geometric and physical constraints for task completion. Across Meta-World, RoboTwin 2.0, and real-world manipulation, CORE improves the average success rate of the corresponding policy backbones by up to +3.9, +11.1, and +17.0 percentage points, respectively, and outperforms text-conditioned variants under the evaluated settings.
