Search papers, labs, and topics across Lattice.
This paper introduces GeoWAM, a visual geometry world action model that utilizes point clouds to represent scene dynamics in autonomous driving, moving away from traditional pixel-based approaches. By pretraining on future scene geometry rather than images, GeoWAM captures spatial structure and temporal evolution more effectively, leading to improved predictions of ego trajectories. Extensive evaluations demonstrate that GeoWAM significantly outperforms image-based models in generating robust driving policies, highlighting the advantages of geometry-based representations in autonomous driving tasks.
Future scene geometry, not pixel data, is the key to unlocking superior driving policies in autonomous systems.
World action models (WAMs) have recently gained increasing attention as a framework for jointly modeling scene evolution and ego actions in autonomous driving. Most existing WAMs learn scene dynamics in pixel space by combining a video-generation backbone for future-observation prediction with an action head for ego-trajectory prediction. Pixels, however, provide only an indirect representation of these dynamics: they entangle geometry and motion with appearance, texture, and illumination, forcing the model to infer three-dimensional transformations from two-dimensional observations. We argue that geometry, represented by point clouds, offers a more natural state space for driving because it explicitly captures spatial structure and the rigid and non-rigid transformations that govern scene evolution while directly aligning with the space in which driving actions are executed. Building on this insight, we introduce \textbf{GeoWAM}, a visual geometry world action model for autonomous driving. Rather than predicting future images, GeoWAM is pretrained to forecast future scene geometry, yielding representations that jointly encode spatial structure and temporal evolution. A geometry-conditioned action head then leverages these learned geometric dynamics to predict future ego trajectories. Extensive open-loop and closed-loop evaluations show that visual geometry world modeling yields substantially stronger driving policies than image-based alternatives, establishing future-geometry prediction as an effective pretraining objective for autonomous driving.
