Search papers, labs, and topics across Lattice.
This paper introduces LaPla, a Vision-Language-Action framework that integrates latent-aligned planning to enhance the execution of continuous actions in autonomous driving. By employing a residual vector-quantized variational autoencoder (VQ-VAE) for action tokenization, LaPla effectively bridges the gap between high-dimensional semantic understanding and the continuous action space, eliminating quantization errors. Experimental results on the nuScenes benchmark show that LaPla outperforms existing methods, achieving a 15.52% reduction in long-horizon L2 error and a 33.34 percentage point improvement in closed-loop driving success rates.
LaPla reduces quantization errors in autonomous driving by transforming high-dimensional semantics into continuous actions, achieving significant performance gains over existing methods.
Bridging the gap between the discrete reasoning of Vision-Language Models and the continuous, physics-constrained nature of autonomous driving remains a significant challenge. In this work, we introduce LaPla, a unified Vision-Language-Action (VLA) framework featuring latent-aligned planning to seamlessly ground semantic understanding in precise motion execution. We first design an action tokenizer based on a residual vector-quantized variational autoencoder (VQ-VAE), capturing vehicle kinematics and encoding trajectory features into a structured latent space. Rather than discrete codebook lookups that inevitably introduce quantization errors, LaPla repurposes this representation as a physical prior to bridge the modality gap between high-dimensional semantics and the raw action space. Specifically, given multimodal inputs integrating multi-view images, historical actions, and textual instructions, LaPla incorporates concurrent action queries to causally attend to the multimodal context in a single forward pass, projecting hidden states directly into the pretrained VQ-VAE latent space. The frozen decoder then translates these continuous latents into actions, effectively eliminating quantization errors and ensuring physically plausible trajectories while bypassing time-consuming autoregressive generation. Extensive experiments on the nuScenes benchmark demonstrate that LaPla achieves competitive open-loop performance, reducing long-horizon L2 error by 15.52% compared to state-of-the-art VLA methods. Closed-loop evaluations on the NVIDIA AlpaSim simulator further confirm its superior capability in ensuring smooth driving progress, improving the success rate by 33.34 percentage points with significantly reduced inference latency.
