Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This survey paper reviews the evolution of end-to-end autonomous driving, contrasting traditional modular systems with emerging end-to-end approaches based on imitation learning, reinforcement learning, and foundation models. It categorizes recent advances in planning, reasoning, data generation, and scene understanding within the context of large language models (LLMs) and vision-language models (VLMs). The paper also addresses challenges like multimodal fusion complexity and safety risks, and outlines future research directions including world models and modular foundation model architectures.
End-to-end autonomous driving is being reshaped by LLMs and VLMs, but challenges in multimodal fusion and safety remain open.
In recent years, the autonomous driving technology has witnessed rapid global development, offering effective solutions to increasingly severe challenges related to traffic congestion and road safety. Among various paradigms, end-to-end autonomous driving has emerged as a promising alternative to traditional modular systems, due to its streamlined architecture, enhanced decision consistency, and superior generalization capabilities. This survey provides a comprehensive review of the evolution and core technologies of end-to-end autonomous driving. It emphasizes the applications and developments of imitation learning (IL), reinforcement learning (RL), and other paradigms. Furthermore, it highlights the emerging paradigm empowered by foundation models, such as large language models (LLMs) and vision–language models (VLMs), and systematically categorizes recent advances in planning, reasoning, data generation, and scene understanding. In light of persistent challenges, such as multimodal fusion complexity, low sample efficiency, and safety risks, this survey summarizes the representative research efforts and corresponding solutions. Finally, it outlines future directions, including the integration of world models (WMs) for unified data generation and inference optimization, the advancement of foundation model architectures toward modular design, sparse activation mechanisms, and knowledge distillation, and the realization of deployable, transferable, and unified multimodal frameworks. This survey aims to serve as a comprehensive theoretical and technical reference for researchers in end-to-end autonomous driving, facilitating its evolution toward higher performance, stronger generalization, and enhanced safety assurance.