Search papers, labs, and topics across Lattice.
This paper introduces a Vision-Language Model (VLM)-based framework for detecting GNSS spoofing in autonomous vehicles by integrating visual data from front cameras with in-vehicle sensor readings. The method employs a three-stage fine-tuning process to align visual and sensor data within a shared semantic space, allowing for effective identification of discrepancies in vehicle maneuvers across various spoofing scenarios. The results demonstrate a significant improvement in detection accuracy, with the fine-tuned model achieving an F1-score of 94%-95%, compared to a baseline of 23%-32%, highlighting its potential as a robust defense mechanism against spoofing attacks.
Achieving up to 95% accuracy in detecting GNSS spoofing attacks, this VLM-based approach transforms how autonomous vehicles can safeguard against navigation threats.
Autonomous vehicles (AVs) depend on Global Navigation Satellite Systems (GNSS) for localization and navigation, making them vulnerable to spoofing attacks that can covertly redirect vehicles or induce unsafe maneuvers. In this paper, we develop the first Vision-Language Model (VLM)-based framework for GNSS spoofing detection for autonomous vehicles by fusing front-camera visual data with in-vehicle sensor readings (e.g., speed, acceleration, yaw rate) against GNSS-derived maneuvers. Our approach introduces a three-stage fine-tuning process that first grounds visual cues, and then calibrates sensor data within a shared semantic space to detect discrepancies between predicted and GNSS-derived maneuvers across three attack scenarios. We also generated an independent real-world dataset by driving an instrumented vehicle on public roads in Tuscaloosa, Alabama, equipped with time-synchronized GNSS, IMU, and camera logs to validate cross-regional generalization of our fine-tuned model on unseen data from training data. On this dataset, we then generated intelligent spoofing attacks, including trajectory mirroring with road-network snapping for wrong-turn attacks, position freezing for overshoot scenarios, and drift generation for stop attacks. On this validation dataset, the zero-shot VLMs baseline F1-score ranges from 23% to 32%, whereas our fine-tuned model achieves an F1-score ranging from 94% to 95%. Results show that our VLM-based approach correctly classified every wrong-turn and stop attacks, and attains 88%-93% accuracy for overshoot attacks. Furthermore, we introduce an adaptive inference policy that reduces VLM invocations to 14% (~86% computational reduction) and yields 65ms-73ms per 4s window. These results point to a practical, on-road layer of defense that complements signal-level integrity checks with the use of VLMs.
