Search papers, labs, and topics across Lattice.
This paper introduces GlanceWAM, a novel world-action model that decouples visual imagination from control to achieve real-time inference while enhancing task success rates. By employing an asynchronous proposer that generates lookahead frames in the background and a control action head that operates at a rapid rate, GlanceWAM effectively mitigates the latency issues associated with synchronous video generation. The model demonstrates superior performance, achieving 72.2% success on the RoboCasa kitchen benchmark, outperforming existing methods while executing actions 24 times faster than synchronous baselines.
GlanceWAM achieves 72.2% success on RoboCasa while executing actions 24 times faster than traditional synchronous models, reshaping the landscape of real-time robot learning.
Video generative models provide rich physical priors for robot learning, yet existing world-action models (WAMs) face a fundamental trade-off: synchronous video generation at control rate is latency-prohibitive, while abandoning test-time visual imagination sacrifices task success. We show that visual imagination achieves both real-time inference and superior success rates when generated asynchronously off the critical path and consumed directly in latent space. We introduce GlanceWAM, which decouples imagination from control within a single video DiT: an asynchronous proposer glances ahead on a slow clock to imagine a single lookahead frame seconds into the future in the background, while an action head decodes action chunks at control rate (48 ms) purely in latent space without blocking. Enabled by a non-interfering attention mask that isolates video representations and staleness-robust horizon training that accommodates asynchronous lookahead aging, GlanceWAM breaks the speed-success dilemma. Trained purely on demonstrations, it attains 72.2% on the 24-task RoboCasa kitchen benchmark (surpassing synchronous Cosmos Policy at 67.1% and imagination-free co-training at 64.4%) and 99.0% on LIBERO, executing at 48 ms per chunk on an NVIDIA A100 GPU (24x faster than synchronous baselines). Code is available at https://github.com/linhanwang/GlanceWAM.
