Search papers, labs, and topics across Lattice.
LVDrive, a new Vision-Language-Action (VLA) framework, enhances autonomous driving by incorporating future scene prediction in a high-level latent space, supervised by a pre-trained vision backbone. Unlike prior work that uses pixel-level image reconstruction, LVDrive jointly models future scene and motion prediction in a unified embedding space. Experiments on Bench2Drive show LVDrive significantly improves closed-loop driving performance compared to action-supervised methods and image-reconstruction-based world models.
Ditch pixel-perfect reconstruction: LVDrive shows that learning future scene representations in a high-level latent space dramatically improves autonomous driving performance.
Vision-Language-Action (VLA) models have emerged as a promising framework for end-to-end autonomous driving. However, existing VLAs typically rely on sparse action supervision, which underutilizes their powerful scene understanding and reasoning capabilities. Recent attempts to incorporate dense visual supervision via world modeling often overemphasize pixel-level image reconstruction, neglecting semantically meaningful scene representation learning. In this work, we propose LVDrive, a Latent Visual representation enhanced VLA framework for autonomous driving. LVDrive introduces a future scene prediction task into the VLA paradigm, where future representations are learned entirely in a high-level latent space under auxiliary supervision from a pretrained vision backbone. Departing from inefficient autoregressive generation, we jointly model future scene and motion prediction within a unified embedding space, processed in a single forward pass to conduct the future-aware reasoning. We further design a two-stage trajectory decoding strategy that explicitly leverages the learned latent future representations to refine trajectory generation. Extensive experiments on the challenging Bench2Drive benchmark demonstrate that LVDrive achieves significant improvements in closed-loop driving performance, outperforming both action supervised methods and image-reconstruction-based world model approaches.
