Search papers, labs, and topics across Lattice.
This study benchmarks the robustness of autonomous driving systems against environmental illusions, such as shadows and reflections, which can significantly impair lane perception. Utilizing the newly introduced LanEvil++ benchmark, the authors evaluate two types of lane perception models鈥攃onventional lane detection and vision-language model-based systems鈥攁cross 94 high-fidelity 3D scenes. The findings reveal that environmental illusions can lead to performance drops of up to 7.20% in accuracy for lane detection models and highlight the critical safety implications through real-world case studies, while the proposed Multimodal Illusion Defense Approach (MIDA) enhances robustness by over 4% in challenging conditions.
Environmental illusions can degrade lane detection accuracy by over 7%, posing serious safety risks for autonomous vehicles.
Environmental illusions (eg., shadows, reflections, and tire marks) are naturally existing yet overlooked phenomena in real-world driving environments. They can disturb visual perception, leading to misinterpretation of the scene and posing serious safety risks to autonomous driving (AD) systems. However, existing researches largely overlook these phenomena, leaving a critical gap. To address this issue, we study AD robustness through the lane perception perspective, a fundamental task supporting core functions like cruise control and lane centering. We focus on two representative models: conventional lane detection (LD) and vision-language model-based systems (ADVLMs). In this work, we introduce the first benchmark, LanEvil++, for evaluating the robustness of lane perception under environmental illusions. LanEvil++ encompasses 14 types of illusions and leverages the CARLA simulator to generate 94 high-fidelity, fully controllable 3D scenes, yielding a dataset of 90,292 annotated images, 1,596 video clips, and 41,855 visual question answering pairs. Extensive evaluations demonstrate that environmental illusions substantially degrade the performance of state-of-the-art LD methods. On average, LD models experience a 5.27% drop in Accuracy and a 10.49% decline in F1-score, while ADVLMs show a 2.03% reduction in GPT-score and a 0.75% drop in Language-score. Among all illusions, shadows emerge as the most disruptive factor, reducing accuracy by up to 7.20%. Furthermore, closed-loop simulations reveal that these illusions can lead to incorrect driving decisions. Complementary real-world case studies highlight safety-critical failures in actual traffic scenes. To enhance robustness, we propose the Multimodal Illusion Defense Approach (MIDA). MIDA achieves substantial gains under challenging conditions, boosting robustness by 4.23% on LD models and 3.82% on ADVLMs.
