Search papers, labs, and topics across Lattice.
This study introduces XEWorld, a cross-embodiment testbed designed to evaluate action-conditioned world models in robotic manipulation by assessing their performance on unseen robot embodiments within identical physical scenes. The findings reveal that current models primarily function as 2D visual pattern matchers, struggling to generalize physical dynamics due to their reliance on visual similarity rather than kinematic properties. The research highlights the necessity for architectural innovations to enable true cross-embodiment generalization, as existing models fail to effectively translate joint actions into coherent visual trajectories and suffer from catastrophic forgetting during few-shot adaptations.
Current action-conditioned world models are limited by their reliance on visual patterns, failing to generalize physical dynamics across different robot embodiments.
Action-conditioned world models are promising learned simulators for robotic manipulation, yet evaluating them exclusively on training robots fails to reveal whether they capture physical dynamics or merely memorize visual patterns. To answer whether a model can faithfully render a robot it has never seen, we introduce XEWorld, a controlled cross-embodiment testbed for world models that isolates embodiments by evaluating held-out robots within physically identical scenes. Our systematic analysis uncovers a shared architectural bottleneck: current models act primarily as 2D visual pattern matchers whose generalization is governed by visual similarity rather than physical kinematic similarity. Driven by this limitation, they struggle to translate abstract numeric joint actions into coherent visual trajectories, and fail to predict dynamic visual changes from static initial observations. Consequently, successfully rendering an unseen embodiment zero-shot strictly requires heavily grounded cues, specifically pixel-space actions and explicit spatial-temporal alignment. Even when bypassing this zero-shot barrier via few-shot adaptation, the forced appearance recovery triggers catastrophic forgetting of seen embodiments. Together, these failures expose a critical inability to apply learned physical dynamics to novel visual appearances, highlighting that achieving true cross-embodiment generalization requires architectural innovations that decouple visual appearance from underlying physical dynamics.
