Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper tackles the challenge of viewpoint-dependent performance in end-to-end autonomous driving models. They introduce a novel augmentation-free approach that leverages a 3D foundation model to provide geometric priors in the form of per-pixel 3D positions as positional embeddings. By fusing intermediate geometric features using cross-attention, the model demonstrates improved robustness to camera viewpoint perturbations, particularly pitch and height variations, on the VR-Drive benchmark.
End-to-end driving models can gain surprising viewpoint robustness simply by injecting 3D geometric priors from a foundation model, without any viewpoint augmentation.
Robust trajectory planning under camera viewpoint changes is important for scalable end-to-end autonomous driving. However, existing models often depend heavily on the camera viewpoints seen during training. We investigate an augmentation-free approach that leverages geometric priors from a 3D foundation model. The method injects per-pixel 3D positions derived from depth estimates as positional embeddings and fuses intermediate geometric features through cross-attention. Experiments on the VR-Drive camera viewpoint perturbation benchmark show reduced performance degradation under most perturbation conditions, with clear improvements under pitch and height perturbations. Gains under longitudinal translation are smaller, suggesting that more viewpoint-agnostic integration is needed for robustness to camera viewpoint changes.