Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces PRISM, a probabilistic deep supervision framework that enhances end-to-end autonomous driving motion planning by directly regularizing intermediate latent representations with ground-truth data. The authors reveal that performance improvements from integrating vision-language model supervision are primarily due to the connections established between the model's latent features and the ground-truth data, rather than the reasoning capabilities of the VLM itself. Evaluations on the nuScenes dataset show that PRISM achieves an 8% reduction in planning L2 error and a 3% decrease in collision rates compared to existing vectorized baselines, demonstrating its efficacy with minimal computational cost.
Ground-truth supervision of latent features can lead to significant performance gains in autonomous driving models, challenging previous assumptions about the role of vision-language models.
End-to-end autonomous driving (E2E AD) systems integrate perception, prediction, and planning into a single differentiable architecture. While these models show great promise, their standard training often relies on output-only supervision, which can lead to weak gradients for the hidden layers of increasingly complex models. Recent works have integrated vision-language model (VLM) supervision for latent features to address this, yielding substantial empirical gains, yet leaving the underlying theoretical mechanisms poorly understood. Our investigation into this methodology reveals that the resulting performance gains stem not from VLM reasoning capabilities, as previously assumed, but rather from the latent connections forged between the E2E AD model and ground-truth (GT) data during training. Building on this insight, we propose a probabilistic deep supervision framework that regularizes intermediate latent representations directly from GT data. By treating model latents as reparameterizable distributions, we optimize the architecture via the Evidence Lower Bound (ELBO). Our evaluations conducted on the nuScenes dataset demonstrate that supervising trajectory-related latents with future GT paths consistently improves planning performance. Using identical training data and E2E architectures, our method achieves an 8% reduction in planning L2 error and a 3% decrease in collision rates compared to competitive vectorized baselines, all while incurring negligible computational overhead.