Search papers, labs, and topics across Lattice.
This paper introduces Enfold, a novel method that internalizes the computation of future-generative models into a predictive representation derived solely from current visual context and language instructions. By training a current-only encoder with multi-level states from a generator, Enfold achieves significant improvements in action prediction efficiency, reducing latency by up to 10.1 times compared to existing methods. The approach not only enhances control in various robotic tasks but also allows for adaptive responses to changes in the environment, challenging traditional fixed trajectory replay models.
Enfold redefines how we leverage world generative models, enabling ultra-efficient control that adapts dynamically to real-time changes in the environment.
World generative models are typically used through what they produce: a rendered future, a video-conditioned action, or latent context computed by a costly generative branch. We argue that their more reusable asset is the computation that constructs a future. As a generator transforms a corrupted future into a coherent trajectory, its intermediate states organize appearance, spatial layout, and interaction across levels of abstraction. Can this future-generative computation be internalized in a representation inferred from the present alone? We present Enfold, which transfers this computation into a representation predicted from the current visual context and language instruction. During training, multi-level states exposed as the generator processes the observed future supervise a current-only encoder. The learned representation is fed back to condition future generation and is read by task heads without allowing task gradients to reshape the encoder. At deployment, action prediction no longer executes the generator. Across LIBERO, RoboTwin2.0, and real-robot tasks, Enfold supports strong control while reducing action latency by 3.7times relative to Fast--WAM, Enfold-Flash reaches 10.1times. Representation analyses show that it suppresses nuisance variation and preferentially captures changes that emerge over longer horizons. When the current scene is altered by human intervention, both the generated continuation and the executed actions adapt, which is inconsistent with fixed trajectory replay. These results recast a world generator as a source of predictive control representations: its future need not be materialized at every step if its internal structure can be enfolded into the present.
