Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
SkeleWAM introduces a novel World Action Model (WAM) that represents manipulation scenes using sparse 3D skeletons of robot joints, object centers, and interaction points, constructed online from RGB-D and proprioception. This geometric state representation enables direct action generation from language instructions and efficient future skeleton prediction, providing implicit geometric supervision for action learning. SkeleWAM achieves state-of-the-art 85.9% success rate on LIBERO-Plus with significantly fewer parameters than prior methods, demonstrating the efficacy of explicit geometric structure for parameter-efficient WAMs.
Explicit 3D skeletal representations unlock parameter-efficient robotic manipulation, achieving 85.9% success on LIBERO-Plus with half the parameters of prior methods.
World action models (WAMs) combine robot action generation with future state prediction. Existing WAMs typically predict videos or learned visual latents, which represent interaction geometry only implicitly and may retain appearance information unrelated to control. We introduce SkeleWAM, a compact WAM that represents a manipulation scene as a sparse 3D skeleton composed of robot joints, object centers, and interaction points. Constructed online from current RGB-D observations and robot proprioception, the skeleton provides a unified geometric state for action generation and future skeleton prediction. Future skeleton prediction provides additional geometric supervision for action learning without requiring visual reconstruction. At inference, SkeleWAM generates actions directly from the current skeleton and language instruction, while Medoid Action Consensus (MAC) serves as an auxiliary consensus strategy for stochastic action samples. On LIBERO-Plus, SkeleWAM achieves an overall success rate of 85.9% with 57.1M parameters, outperforming Cosmos-Policy by 3.7 percentage points. These results demonstrate that sparse 3D robot--object structure provides an effective state space for robust and parameter-efficient world action learning. The project is available at https://skelewam-project.github.io/.