Search papers, labs, and topics across Lattice.
This paper introduces MECo-WAM, a Multi-Expert Co-Training World Action Model that integrates action-relevant 4D geometric priors into video-action representations to enhance robotic manipulation. By employing a combination of video and action experts alongside a lightweight 4D expert, the model captures the temporally evolving geometry essential for precise manipulation while maintaining a lightweight inference graph. Experimental results demonstrate that MECo-WAM significantly improves manipulation performance on various benchmarks without increasing inference costs, achieving state-of-the-art results on LIBERO and RoboTwin 2.0.
Injecting 4D geometric priors into World Action Models boosts robotic manipulation performance without adding inference overhead.
World Action Models (WAMs) have shown strong potential for robotic manipulation by jointly modeling visual future dynamics and executable action sequences. However, existing video-action co-training methods primarily optimize appearance-oriented video latents, which may insufficiently capture the temporally evolving geometry required for precise manipulation. We propose MECo-WAM, a Multi-Expert Co-Training World Action Model that injects action-relevant 4D geometric priors into video-action representations while preserving the original lightweight inference graph. During training, MECo-WAM combines video and action experts with a lightweight 4D expert supervised by relational targets from a frozen VGGT encoder. Asymmetric expert visibility prevents non-causal shortcuts from auxiliary geometry to action generation. To transfer geometric knowledge into the deployed video-action pathway, we introduce decayed 4D read-mask attention, which provides restricted current-frame geometric guidance early in training and progressively removes this dependency. We further propose action-aware temporal geometric distillation, which aligns within-frame geometric relations and their temporal evolution while emphasizing visual regions most relevant to robot actions. At deployment, all auxiliary 4D components are removed. Experiments on LIBERO (98.2%), RoboTwin 2.0 (92.6%), and challenging real-world manipulation tasks show that MECo-WAM improves manipulation performance without increasing inference cost.
