Search papers, labs, and topics across Lattice.
This survey paper analyzes object detection techniques for autonomous vehicles, focusing on the integration of multimodal perception, contextual reasoning, and cooperative intelligence. It reviews AV sensors (camera, ultrasonic, LiDAR, and Radar) and their fusion strategies, along with a structured categorization of AV datasets. The paper emphasizes emerging transformer-driven approaches powered by Vision Transformers (ViTs), Large Language Models (LLMs), Small Language Models (SLMs), and Vision-Language Models (VLMs) to provide a roadmap of current capabilities, challenges, and future opportunities in the field.
Autonomous vehicle object detection is poised for a revolution, with LLMs and VLMs poised to fuse multimodal sensor data and cooperative intelligence for unprecedented reliability.
Autonomous Vehicles (AVs) are transforming the future of transportation through advances in intelligent perception, decision-making, and control systems. However, their success is tied to one core capability, reliable object detection in complex and multimodal environments. While recent breakthroughs in Computer Vision (CV) and Artificial Intelligence (AI) have driven remarkable progress, the field still faces a critical challenge as knowledge remains fragmented across multimodal perception, contextual reasoning, and cooperative intelligence. This survey bridges that gap by delivering a forward-looking analysis of object detection in AVs, emphasizing emerging paradigms such as Vision-Language Models (VLMs), Large Language Models (LLMs), and Generative AI rather than re-examining outdated techniques. We begin by systematically reviewing the fundamental spectrum of AV sensors (camera, ultrasonic, LiDAR, and Radar) and their fusion strategies, highlighting not only their capabilities and limitations in dynamic driving environments but also their potential to integrate with recent advances in LLM/VLM-driven perception frameworks. Next, we introduce a structured categorization of AV datasets that moves beyond simple collections, positioning ego-vehicle, infrastructure-based, and cooperative datasets (e.g., V2V, V2I, V2X, I2I), followed by a cross-analysis of data structures and characteristics. Ultimately, we analyze cutting-edge detection methodologies, ranging from 2D and 3D pipelines to hybrid sensor fusion, with particular attention to emerging transformer-driven approaches powered by Vision Transformers (ViTs), Large and Small Language Models (SLMs), and VLMs. By synthesizing these perspectives, our survey delivers a clear roadmap of current capabilities, open challenges, and future opportunities.
