Search papers, labs, and topics across Lattice.
This report details significant enhancements to AlayaWorld, focusing on the integration of conditioning signals to better align with generated content in both latent representation and temporal structure. Key modifications include the introduction of a streaming 3D point-cache renderer and a redesigned conditioning pipeline that utilizes motion-aware latent conditioning, resulting in improved coherence and realism in long-horizon world modeling. The new architecture demonstrates a marked improvement in the model's ability to generate temporally consistent and visually coherent video sequences.
Conditioning signals now match generated content more closely, leading to unprecedented realism in long-horizon video generation.
This report presents an improved version of AlayaWorld. While the backbone architecture, chunk-wise autoregressive generation scheme, and training data remain unchanged from the previous release, we substantially revise how conditioning signals are represented and integrated into the model. The new design is guided by a simple principle: conditioning signals should match the generated content as closely as possible in both latent representation and temporal structure. To this end, we make two major changes. First, we replace the previous depth-warping-based spatial memory with a streaming 3D point-cache renderer. Second, we redesign the conditioning pipeline so that visual conditions are encoded in the same causal-VAE latent space, with temporal statistics consistent with those of the generated video. Concretely, the new version introduces six modifications: (1) replacing static-frame image conditioning with motion-aware latent conditioning; (2) causally encoding re-rendered spatial memory as a continuous sequence; (3) aligning the temporal-memory window in pixel space; (4) adopting hard memory dropout that removes memory tokens rather than zeroing them; (5) unifying the VAE encoding and decoding protocol across training and inference; and (6) removing the camera AdaLN branch, such that viewpoint control is provided entirely through the re-rendered spatial condition.
