Search papers, labs, and topics across Lattice.
This paper introduces S$^2$VAE, a variational autoencoder that uses a product of Power Spherical latent distributions to encode geometric information from Visual Geometry Grounded Transformer (VGGT) features. By enforcing hyperspherical structure in the latent space, S$^2$VAE aims to better preserve 3D geometry and camera dynamics compared to appearance-only models. Experiments on depth estimation, camera pose recovery, and point cloud reconstruction demonstrate that S$^2$VAE outperforms Gaussian bottlenecks, especially under high compression.
Hyperspherical latent spaces unlock better 3D scene understanding from vision transformers, especially when bandwidth is constrained.
Modern visual world modeling systems increasingly rely on high-capacity architectures and large-scale data to produce plausible motion, yet they often fail to preserve underlying 3D geometry or physically consistent camera dynamics. A key limitation lies not only in model capacity, but in the latent representations used to encode geometric structure. We propose S$^2$VAE, a geometry-first latent learning framework that focuses on compressing and representing the latent 3D state of a scene, including camera motion, depth, and point-level structure, rather than modeling appearance alone. Building on representations from a Visual Geometry Grounded Transformer (VGGT), we introduce a novel type of variational autoencoder using a product of Power Spherical latent distributions, explicitly enforcing hyperspherical structure in the bottleneck to preserve directional and geometric semantics under strong compression. Across depth estimation, camera pose recovery, and point cloud reconstruction, we show that geometry-aligned hyperspherical latents consistently outperform conventional Gaussian bottlenecks, particularly in high-compression regimes. Our results highlight latent geometry as a first-class design choice for physically grounded visual and world models.
