Search papers, labs, and topics across Lattice.
GigaWorld-Policy-0.5 enhances World Action Models (WAMs) by adopting an action-centered formulation that utilizes future visual dynamics during training while allowing action-only decoding at inference, significantly reducing computational overhead. This approach leverages a Mixture-of-Transformers architecture to separate visual dynamics modeling from action generation, achieving an impressive 85 ms inference latency on an RTX 4090 setup. Experimental results demonstrate that this method maintains the advantages of future visual dynamics while improving efficiency in real-time robot control applications.
GigaWorld-Policy-0.5 achieves real-time robot control with 85 ms inference latency by decoupling visual dynamics from action generation, revolutionizing WAM efficiency.
World Action Models (WAMs) improve robot policy learning by jointly modeling actions and future visual observations, using future scene evolution as dense supervision for physically grounded action generation. However, a common design in existing WAMs is to explicitly generate future videos at inference time, incurring substantial computational overhead and hindering real-time closed-loop deployment. GigaWorld-Policy addresses this issue with an action-centered formulation, where future visual dynamics are used during training while action-only decoding is used at inference time. Building upon this framework, we present GigaWorld-Policy-0.5, an enhanced action-centered WAM designed for more efficient robot control. During pretraining, GigaWorld-Policy-0.5 adopts a mixed Action-Conditioned World Modeling (AC-WM) and WAM training strategy. This strengthens the coupling between visual dynamics and robot actions and improves the transferability of action representations for downstream policy learning. For efficient inference, GigaWorld-Policy-0.5 introduces a Mixture-of-Transformers architecture that separates visual dynamics modeling and action generation into specialized experts, reducing active computation during action-only inference and achieving 85 ms inference latency on a local RTX 4090 setup. In addition, we employ an agent-based AutoResearch pipeline to systematically search training configurations, enabling more efficient identification of optimal experimental setups while reducing the time and manual intervention required for hyperparameter tuning. Experiments and ablations show that GigaWorld-Policy-0.5 preserves the training benefits of future visual dynamics while improving inference efficiency for robot control.
Tsinghua AI