Search papers, labs, and topics across Lattice.
This paper introduces SV-WAM, a surround-view world-action model designed for end-to-end autonomous driving, which efficiently utilizes full six-camera observations while minimizing computational overhead during inference. By employing future-video prediction as training supervision and implementing an action-centered causal mask, SV-WAM allows for effective action planning without the need for video generation at deployment. Experimental results on the NAVSIMv2 and nuScenes benchmarks show that SV-WAM achieves state-of-the-art planning performance with low latency and strong zero-shot transfer capabilities.
SV-WAM achieves high-performance autonomous driving planning without the computational burden of generating future videos during inference.
World models (WMs) have demonstrated strong potential for end-to-end autonomous driving by learning predictive representations of future scene dynamics. However, generating future videos during inference introduces substantial computational overhead, leading many recent driving WMs to adopt a single front camera as input for efficient deployment. This design restricts spatial coverage in safety-critical maneuvers such as lane changes, merges, and turns. To address this limitation, we propose SV-WAM, a surround-view world-action model (WAM) that preserves full six-camera observations while maintaining efficient inference. SV-WAM leverages future-video prediction as dense training supervision for action learning within a shared generative model, rather than as an inference-time output. At the core of this design is an action-centered causal mask that prevents action tokens from attending to future-video tokens during joint action-video denoising. Consequently, the video branch can be discarded at deployment, enabling efficient action-only planning. Furthermore, we introduce a differentiable drivable-area compliance regularizer that penalizes vehicle-footprint corners approaching or crossing drivable boundaries, improving planning safety and boundary awareness. Extensive experiments on the closed-loop NAVSIMv2 benchmark and the open-loop nuScenes benchmark demonstrate that SV-WAM achieves state-of-the-art planning performance with low inference latency and competitive zero-shot transfer capability.
